Skip to content

Onetail/Crawler-with-kafka-docker

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Crawler-with-kafka-docker

can crawler yahoo news and analysis

  • python main.py

run this process

to run crawler and kafka producer and consumer data

  • python main.py consumer

consumer get data

see consumer get data

This is use kafka-docker for message queue and python crawler get data doing analysis cosine similarity for top 5 csv content csv content detail top 5 news

About

homework to crawler and anaylsis

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages