最近要租房子,用Python看一下房源吧..

2023-01-05 13:45:37

前言：最近我的朋友想要租房子,为了装个b,决定运用技术去帮助他。

这个网站是什么我也不知道反正是一个房子交易网站 http://www.ljia.net/

设置请求头

headers = {'Accept': '*/*',

               'Accept-Language': 'en-US,en;q=0.8',

               'Cache-Control': 'max-age=0',

               'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36',

               'Connection': 'keep-alive',

               'Referer': 'http://www.baidu.com/'

               }

代码：

import requests

from bs4 import BeautifulSoup

import re;

headers = {'Accept': '*/*',

               'Accept-Language': 'en-US,en;q=0.8',

               'Cache-Control': 'max-age=43200',

               'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36',

               'Connection': 'keep-alive',

               'Referer': 'no-referrer-when-downgrade'

               }

def get_Detail_List(pageindex):

    url = "http://www.ljia.net/zf/p-"+str(pageindex)+'.html'

    response = requests.get(url)

    soup = BeautifulSoup(response.text,'html.parser')

    resultTitle = soup.find_all('h2',{'class','title'})

    for i in resultTitle:

        urler = 'http://www.ljia.net'+i.a['href']

        response = requests.get(urler)

        souper = BeautifulSoup(response.content)

        content = souper.select('div #lpname div[class=floatl]')

        dj = BeautifulSoup(str(content))

        money = souper.find_all('div',{'class','trl-item'})

        moneybs = BeautifulSoup(str(money))

        xinxiList = souper.find_all('div',{'class','tt'})

        xinxi = BeautifulSoup(str(xinxiList))

        consolestr = dj.get_text().replace('[','').replace(']','').strip()+moneybs.get_text().replace('[','').replace(']','').strip()+xinxi.get_text().replace('[','').replace(']','').strip()

        print(consolestr.strip().replace(' ', '').replace('\n', '').replace('\t', '').replace('\r', '').strip())

count = 0

while(count<100):

    get_Detail_List(count)

    count = count+1

哈哈就是这么简单,让我们看下效果图吧！！！

然后,你可以把这些数据放到Mysql,Redis 那我可管不着都可以的了

import pymysql.cursors

# 连接MySQL数据库

connection = pymysql.connect(host='127.0.0.1', port=3306, user='root', password='', db='guest',

                             charset='utf8mb4', cursorclass=pymysql.cursors.DictCursor)

# 通过cursor创建游标

cursor = connection.cursor()

# 创建sql 语句，并执行

sql = "INSERT INTO `users` (`email`, `password`) VALUES ('huzhiheng@itest.info', '123456')"

cursor.execute(sql)

# 提交SQL

connection.commit()

回头在搞个图形！！睡觉！

码农公寓

相关文章