抓取（爬取）网上信息的脚本程序，俗称网络蜘蛛。
powershell中自带了这样的两个命令，【Invoke-WebRequest】和【Invoke-RestMethod】，但这两个命令有时候会乱码。
现在转帖分享， 某个【歪果仁】写的脚本。来源于 墙外出处： https://gist.github.com/angel-vladov/9482676
核心代码
function Read-HtmlPage {
param ([Parameter(Mandatory=$true, Position=0, ValueFromPipeline=$true)][String] $Uri)
# Invoke-WebRequest and Invoke-RestMethod can't work properly with UTF-8 Response so we need to do things this way.
[Net.HttpWebRequest]$WebRequest = [Net.WebRequest]::Create($Uri)
[Net.HttpWebResponse]$WebResponse = $WebRequest.GetResponse()
$Reader = New-Object IO.StreamReader($WebResponse.GetResponseStream())
$Response = $Reader.ReadToEnd()
$Reader.Close()
# Create the document class
[mshtml.HTMLDocumentClass] $Doc = New-Object -com "HTMLFILE"
$Doc.IHTMLDocument2_write($Response)
# Returns a HTMLDocumentClass instance just like Invoke-WebRequest ParsedHtml
$Doc
#powershell 传教士 转帖并修改的文章 2016-01-01, 允许再次转载，但必须保留名字和出处，否则追究法律责任
}
原文函数
function Read-HtmlPage {
param ([Parameter(Mandatory=$true, Position=0, ValueFromPipeline=$true)][String] $Uri)
# Invoke-WebRequest and Invoke-RestMethod can't work properly with UTF-8 Response so we need to do things this way.
[Net.HttpWebRequest]$WebRequest = [Net.WebRequest]::Create($Uri)
[Net.HttpWebResponse]$WebResponse = $WebRequest.GetResponse()
$Reader = New-Object IO.StreamReader($WebResponse.GetResponseStream())
$Response = $Reader.ReadToEnd()
$Reader.Close()
# Create the document class
[mshtml.HTMLDocumentClass] $Doc = New-Object -com "HTMLFILE"
$Doc.IHTMLDocument2_write($Response)
# Returns a HTMLDocumentClass instance just like Invoke-WebRequest ParsedHtml
$Doc
}
PowerShell function you can use for reading UTF8 encoded HTML pages content. The built in Invoke-WebRequest and Invoke-RestMethod fail miserably.
免责声明：本站文章均来自网站采集或用户投稿，网站不提供任何软件下载或自行开发的软件！
如有用户或公司发现本站内容信息存在侵权行为，请邮件告知！ 858582#qq.com
内蒙古资源网 Copyright www.nmgbbs.com
暂无“powershell网络蜘蛛解决乱码问题”评论...