ANALYSING WALLSTREETBETS WITH PYTHON (REDDIT API)
GitHub Code: https://github.com/georgefree123/Wall...
Medium Write-Up: / analysing-wallstreetbets-with-python
NLTK Article: https://www.digitalocean.com/communit...
---
The popularity of WallStreetBets has gone through the roof in recent months. What started out as a fringe subreddit has become one of the most talked-about “financial” sites in the world.
Although there are a lot more busts than booms, thanks to the GME saga even hedge-funds are now taking notice of what people are rallying behind on the subreddit.
And if it’s good enough for them, surely it’s good enough for us?
Before I go any further I want to emphasise that 99% of the posts on WSBs are junk, and anything you glean from it should only act to focus your reading - it would be stupid to make a trade based off of WSBs alone. And therefore, the same stands for the code we’re going through.
Now, if we wanted to assess what’s hot on the WSBs subreddit we could manually go through the most recent posts, and try and work out an understanding of what’s going on, and the general sentiment of the post and all of the associated comments. Clearly this isn’t a very effective or accurate method though.
Really what we want is to automate the process, at least in part. Broadly speaking there are two ways we can do this. The first is to scrape the subreddit using something like Beautiful Soup. The downside to this approach is that the code becomes pretty complicated and some of the comments can be missed. Arguably a better solution is to use the Reddit API.
Just a heads up, this video isn’t a comprehensive coding tutorial. I’m just going to walk through the main aspects of the code. The full code is linked to in the description.
In short, the code downloads the most recent posts and associated comments using the Reddit API, extracts any tickers in the comments and post body, and use the NLTK library to judge the sentiment of the comments and posts. This is then all outputted to a series of CSVs.
Okay, so if we have a quick look over the code we can see that there are five main areas. The first is grabbing posts from the Reddit API. We do this using the /new endpoint in the get_recent_posts() function. Once we have the posts we then need to parse them, and we do that using the ParsePost function inside of the ParsePost class. Essentially, this just extracts the elements we’re interested in from the JSON returned from the API endpoint, and discards the rest. The useful information is then returned.
Next up we handle comments. Again, comments are sourced using the Reddit API, this time using the /comments/articles endpoint. To make a call all we have to do is provide the ID of a post we want the comments for.
Unlike the post data, parsing the comments JSON is complicated. This is because comments can be made on comments, resulting in potentially very deeply nested JSON. To overcome this we use a set of functions inside of the ParseComments class, which ensures we get every reply that’s available - no matter how many comments come before that. One thing to note, however, is that there is a cut off to how many comments are sent and received using the /comments/articles endpoint. Once a certain limit is reached further comments have to be requested using a different POST endpoint. Because we’re only interested in the overall sentiment of a set of comments, we’re going to overlook these missing comments.
Now that we have all of the raw data we need, it’s time to start extracting meaning. There are two parts to this, the first being listing all of the tickers present. We achieve this through the Tickers class. However, this is dependent on CSVs from the Nasdaq website, which provide information on tickers for all of the major US exchanges. Make sure you download all of the CSVs you want into the relevant directory before running the code.
The Tickers class begins by loading in all of the CSVs found in the tickers directory, and creating a list containing the relevant information. Once this is completed we can call the check_for_tickers() function to identify all of the tickers present in the provided string.
The second part of meaning extraction takes the form of sentiment analysis. The code used here is based heavily off of Date-yaris’ post on the Digital Ocean Community pages. I’ll post a link to the piece in the description.
It’s worth noting that the code isn’t optimised for use with WallStreetBets, or financial data in general - it was originally built to assess the sentiment of Tweets, and uses a Tweet dataset. So don’t rely on it.
Essentially, when the user inputs a string to the NLTK model a “positive” or “negative” result is returned. Whilst this may seem rudimentary, we can use it to determine if a discussion is positive or negative, guiding our further reading.
Sur cette page du site, vous pouvez voir la vidéo en ligne ANALYSING WALLSTREETBETS WITH PYTHON (REDDIT API) durée heure minute seconde en bonne qualité , qui a été Téléchargé par l'utilisateur George Free 06 mars 2021, Partagez le lien avec vos amis et connaissances, sur youtube cette vidéo a déjà été regardée 424 fois et il a aimé 6 téléspectateurs. Bon visionnage!
