ruby爬虫爬取网页源码代码

代码语言:ruby

所属分类:

代码描述:ruby爬虫爬取网页源码代码

代码标签: 网页 源码

下面为部分代码预览,完整代码请点击下载或在bfwstudio webide中打开

require "net/https"  
require "uri"  
  
uri = URI.parse("http://bfw.wiki")  
http = Net::HTTP.new(uri.host, uri.port)  #new a http
request = Net::HTTP::Get.new(uri.request_uri)  #then,new a request
request.initialize_http_header({"User-Agent" => "My Ruby Script"})  
  
response = http.request(request)  
puts response.code  
# => 302  
puts response["location"] # All headers are lowercase  
# => http://www.bfw.wiki/

#post request


uri = URI.parse("http://www.bfw.wiki/")
# Shortcut
response = Net::HTTP.post_form(uri, {"q" => "My query", "per_page" => "50"})

# Full control
#http = Net::HTTP.new(uri.host, uri.port)
#request = Net::HTTP::Post.new(uri.request_uri)
#request.set_form_data({"q" => "My query", "per_page" => "50"})
#response = http.request(request)

# save and use cookie
#http = Net::HTTP.new(uri.host,uri.port)
#path = "/login.html"
#cookie = response["set-cookie"]
#data = "key1=value1&key.........完整代码请登录后点击上方下载按钮下载查看

网友评论0