Hey how are you all. Today I am announcing the test release of Bollywood analytics dot com. You can visit the site here.
Note that this version is just testing purpose and there are just few graphs showing the count of tweets according to movies. I am working on it and will be updating with the better UI version in coming months.
Why I build it?
It was just for fun and learning and exploring different possibilities with twitter streaming apis.
What I used?
Major components are twitter streaming apis, python and flask for backend, very simple JS for front end. Supervisors for tasks running that gets the tweets and put it into Queues to be processed by the processor worker which runs every 20 min and refresh the data on site. High charts and google charts for the charts, redis for datastore and uwsgi for server.
What I actually did?
I wrote a script which uses the twitter streaming apis and push the tweets to the queue. I ran the script for different movies and used supervisor to manage them.
Now I wrote a worker which gets the tweet, read it get the previous count append to it and save it in redis.
Next I add a simple flask server which get the data and returns it.
Next used the high charts and google charts to use the apis and return the results.
Thats it then I ran the server using uwsgi with 5 workers which keeps serving the requests.
How much data processed till now?
Till now I have processed half a million tweets for the movies listed.
Future Scope
I am planning to use the data to make some kind of prediction like what could be the earning of the movies or something like that. Will update once I come up with something better.
Thanks for the read. Share if you like.
Comment below what you would like to see on bollywoodanalytics.com
Thanks.
