文章目录
分享文章
制作采集站时,需要批量采集内容,因此需要根据关键词生成大量的bing链接,然后使用火车头进行采集,在整理好关键词之后,可以使用该脚本批量生成:
import re
def get_bing_url(keywords):
keywords = keywords.strip('\n')
bing_url = re.sub(r'^', 'https://www.bing.com/search?q=', keywords)
bing_url = re.sub(r'\s', '+', bing_url)
return bing_url
if __name__ == '__main__':
with open('keywords.txt', 'r', encoding='utf-8') as f:
lists = f.readlines()
new_lists = list(set(lists))
for list in new_lists:
bing_url = get_bing_url(list)
with open('news_keywords.txt', 'a', encoding='utf-8') as f:
f.write(bing_url.strip() + '\n')