This function removes the carriage returns in the text, removes the text between brackets (e.g. In order to clean up the text, we’ll adapt a function we’ve used in previous posts. “ Nas,” “ Grand Wizard + Nas”), and the song structure (“ Intro,” “ Hook,” etc.). “ Faith N & Nas” in the first line), the artist (e.g. There are carriage returns (indicated by “ ”), information about the producers (e.g.

The head of this dataset can be seen below:ĭata Pre-Processing & Exploratory Analysis Text CleaningĪs can be seen in the above screenshot, the text is pretty messy. My scraping program returned 340 songs in total our original raw dataset therefore contains this many rows. The second is the column called “ rapper” which can take on two values: “Nas” or “DOOM.” The first is the column containing the song lyrics, called “ lyrics_clean” (though, as we’ll see below, this field still requires some serious cleaning before being ready for analysis). For the purposes of the present analysis, we will focus on two main columns in our dataset.

The lyric dataset contains 1 line for each song. Beautiful Soup works really well, is relatively easy to use, and made the task of obtaining these data very straightforward. Although I won’t go into the details in this blog post, I used the Python module Beautiful Soup to scrape the lyrics. The lyrics were scraped from, a website that allows users to transcribe and annotate song lyrics. Madvillainy, The Mouse and the Mask, Key to the Kuffs, etc.) are included. Note that albums of instrumentals and work released prior to the adoption of the DOOM persona are not considered, while albums created with collaborators (e.g. FOOD, The Mouse and The Mask, Born Like This, Key to the Kuffs, and NehruvianDOOM. I included the following DOOM albums: Operation: Doomsday., Take Me to Your Leader, Vaudeville Villain, Madvillainy, Venomous Villain, MM. For Nas, the following albums were included: Illmatic, It Was Written, I Am…, Nastradamus, Stillmatic, The Lost Tapes, God’s Son, Street’s Disciple, Hip Hop is Dead, Untitled, and Life Is Good. This analysis was undertaken in the beginning of November 2017, and all albums released at that point were used. I considered all of the text from official studio albums for each artist (list created by consulting the respective artist Wikipedia pages). Nas or DOOM) of a song, given the song lyrics and 3) use the results of our model to understand the linguistic choices that differentiate the two artists. The goal of the analysis presented in this post will be to: 1) clean and explore the text data 2) build a model to predict the rapper (e.g. Specifically, in this post we will be looking at the collective work, in the form of the songs from all official studio albums, of two elder rap statesmen: Nas and DOOM.

In this post, we’ll return to analyzing rap lyrics using statistical and data analytic tools (the first posts of this blog dealt primarily with this topic).

Doom born like this zip