Rolling Stone published this recent story (https://www.rollingstone.com/pro/features/spotify-sleep-music-playlists-lady-gaga-1223911/) about the streaming success of the sleep-noise artist/label/scheme Sleep Fruits, who chop up background rain-noise recordings into :30 lengths to maximize streaming playcounts.  

Sleep Fruits is undeniably and intentionally exploiting the systemic weakness of the industry-wide :30-or-more-is-a-play rule, as too are audiobook licensors who split their long content into :30 "chapters". The :30 thing is a bad rule. Most of the straightforward alternatives are also bad, so it wasn't an obviously insane initial system design-choice, but this abuse vector is logical and inevitable.  

The effect of the abuse for the label doing it is simple: exploitative multiplication of their "natural" streams by a large factor. x6 if you compare it to rain noise sliced into pop-song-size lengths.  

The effect on the rest of the streaming economy is more complicated. More money to Sleep Fruits does mean less money to somebody else, at least in the short term.  

Under the current pro-rata royalty-allocation system used by all major subscription streaming services (one big pool, split by stream-share), the effects of Sleep Fruits' abuse are distributed across the whole subscription pool. The burden is shared by all other artists, collectively, but is fractional and negligible for any individual artist. In addition, under pro-rata if an individual listener plays Sleep Fruits overnight, every night, it doesn't change the value of their "real" music-listening activity during the day. Those artists get the same benefit from those fans as they would from a listener who sleeps in silence.  

Under the oft-proposed user-centric payment system, in which each listener's payments are split according to only their plays, Sleep Fruits' short-track abuse tactic would be less effective for them. Not as much less effective as you might think, because the same two things that inflate their overall numbers (long-duration background playing + short tracks) inflate their share of each listener's plays. But less, because in the pro-rata model one listener can direct more revenue than they contributed, and in the user-centric model they can't.  

In the user-centric model, though, if an individual listener listens to Sleep Fruits overnight, that directly reduces the money that goes to their daytime artists. Where pro-rata disperses the burden, user-centric would concentrate it on the kinds of artists whose fans also listen to background noise. This is probably worse in overall fairness, and it's definitely worse in terms of the listener-artist relationship, which is one of the key emotional arguments for the user-centric model.  

The interesting additional economic twist to this particular case, though, is that sleeping to background noise works very badly if it's interrupted by ads. Background listening is thus a powerful incentive for paid subscriptions over ad-supported streaming. (Audiobooks similarly, since they essentially require full on-demand listening control.) So if Sleep Fruits drives background listeners to subscribe, it might be bringing in additional money that could offset or even exceed the amount extracted by its abuse. (Maybe. The counterfactual here is hard to assess quantitatively.)  

And although the :30 rule is what made this example newsworthy in its exaggerated effect, in truth it's probably not really the fundamental problem. The deeper issue is just that we subjectively value music based on the attention we pay to it, but we haven't figured out a good way to translate between attention paid and money paid. Switching from play-share to time-share would eliminate the advantage of cutting up rain noise into :30 lengths, but wouldn't change the imbalance between 8 hours/night of sleep loops and 1-2 hours/day of music listening. CDs "solved" this by making you pay for your expected attention with a high fixed entry price, which isn't really any more sensible.  

I don't think we're going to solve this with just math, which disappoints me personally, since I'm pretty good at solving math-solvable things with math. But in general I think time-share is a slightly closer approximation of attention-share than play-share, and thus preferable. And rather than trying to discount low-attention listening, which seems problematic and thankless and negative, it seems more practical and appealing to me to try to add incremental additional rewards to high-attention fandom. E.g. higher-cost subscription plans in which the extra money goes directly to artists of the listener's choice, in the form of microfanclubs supported by platform-provided community features. There are a lot of people who, like me, used to spend a lot more than $10/month on music, and could probably be convinced to spend more than that again if there were reasons.  

Of course, not coincidentally, I have ideas about community features that can be provided with math. Lots of ideas. They come to me every :30 while I sleep.  
 

PS: I've seen some speculation that Sleep Fruits is buying their streams. I'm involved enough in fraud-detection at Spotify to say with at least a little bit of confidence that this is probably not the case. Large-scale fraud is pretty easy to detect, and the scale of this is large. It's abusing a systemic weakness, but not obviously dishonestly.

The 2020 Pop Conference starts this week! It's online and free, so you can come!  

But you do need to register!  

My PopCon contribution this year is not a talk, it's a web explorer for the music that defines, unites and distinguishes kids around the world. It's called The Aqueduct of Youth.  

 

It's part of a panel called The Platforms of Youth: Meme-ing, Marketing & Streaming, with five other explorations of "Old Town Road", Christian musicals, aging, Tiktok and VSCO Girls:  

 

You can watch videos from the other participants, drink from the Aqueduct, and then come and ask us questions in our live Zoom discussion session on PopCon's opening day, Wednesday 9/9, from 10:30-11 Pacific / 1:30-2pm Eastern.

I starting making one music-list a year some time in the 80s, before I really knew enough for there to be any sense to this activity. For a while in the 90s and 00s I got more serious about it, and statistically way better-informed, but there's actually no amount of informedness that transforms a single person's opinions about music into anything that inherently matters to anybody other than people (if any) who happen to share their specific tastes, or extraordinarily patient and maybe slightly creepy friends.  

Collect people together, though, and the patterns of their love are sometimes very interesting. For several years I presided computationally over an assembly of nominal expertise, trying to find ways to turn hundreds of opinions into at least plural insights. Hundreds of people is not a lot, though, and asking people to pretend their opinions matter is a dubious way to find out what they really love. I'm not really sad we stopped doing that.  

Hundreds of millions of people isn't that much, yet, but it's getting there, and asking people to spend their lives loving all the innumerable things they love is a more realistic proposition than getting them to make short numbered lists on annual deadlines. Finding an individual person who shares your exact taste, in the real world, is not only laborious to the point of preventative difficulty, but maybe not even reliably possible in theory. Finding groups of people in the virtual world who collectively approximate aspects of your taste is, due to the primitive current state of data-transparency, no easier for you.  

But it has been my job, for the last few years, to try to figure out algorithmic ways to turn collective love and listening patterns into music insights and experiences. I work at Spotify, so I have extremely good information about what people like in Sweden and Norway, fairly decent information about most of the rest of Europe, the Americas and parts of Asia, and at least glimmers of insight about literally almost everywhere else on Earth. I don't know that much about you, but I know a little bit about a lot of people.  

So now I make a lot of lists. Here, in fact, are algorithmically-generated playlists of the songs that defined, united and distinguished the fans and love and new music in 2000+ genres and countries around the world in 2019:  

2019 Around the World
 

You probably don't share my tastes, and this is a pretty weak unifying force for everybody who isn't me, but there are so many stronger ones. Maybe some of the ones that pull on you are represented here. Maybe some of the communities implied and channeled here have been unknowingly incomplete without you. Maybe you have not yet discovered half of the things you will eventually adore. Maybe this is how you find them.  
 

I found a lot of things this year, myself, some of them in this process of trying to find music for other people, and some of them just by listening. You needn't care about what I like. And if for some reason you do, you can already find out what it is in unmanageable weekly detail. But I like to look back at my own years. Spotify's official forms of nostalgia so far define years purely by listening dates, but as a music geek of a particular sort, what I mean by a year is music that was both made and heard then. New music.  

I no longer want to make this list by applying manual reductive retroactive impressions to what I remember of the year, but I also don't have to. Adapting my collective engines to the individual, then, here is the purely data-generated playlist of the new music to which I demonstrated the most actual listening attachment in 2019:  

2019 Greatest Hits (for glenn mcdonald)  
 

And for segmented nostalgia, because that's what kind of nostalgist I am, I also used genre metadata and a very small amount of manual tweaking to almost automatically produce three more specialized lists:  

Bright Swords in the Void (Metal and metal-adjacent noises, from the floridly melodic to the stochastically apocalyptic.)
Gradient Dissent (Ambient, noise, epicore and other abstract geometries.)
Dancing With Tears (Pop, rock, hip hop and other sentimental forms.)  
 

And finally, although surely this, if anything, will be of interest to absolutely nobody but me, I also used a combination of my own listening, broken down by genre, and the global 2019 genre lists, to produce a list of the songs I missed or intentionally avoided despite their being popular with the fans of my favorite genres.  

2019 Greatest Misses (for glenn mcdonald)  

I made versions of this Misses list in November and December, to see what I was in danger of missing before the year actually ended, so these songs are the reverse-evolutionary survivors of two generations of augmented remedial listening. But I played it again just now, and it still sounds basically great to me. I'm pretty sure I could spend the next year listening to nothing but songs I missed in 2019 despite trying to hear them all, and it would be just as great in sonic terms. There's something hypothetically comforting in that, at least until I starting trying to figure out what kind of global catastrophe I'm effectively imagining here. I'm alive, but all the musicians in the world are dead? Or there's no surviving technology for recording music, but somehow Spotify servers and the worldwide cell and wifi networks still work?  

Easier to live. I now declare 2019 complete and archived. Onwards.

One of the questions we've yet to exactly answer, about the new streaming-based music business, is how you get started in it. In the old business, you mostly got started by playing your music for people near you. The new one has the potential to be strictly better than this, it seems to me, both by giving you more power to reach the people around you even when you aren't on a stage in one of their bars, and by giving listeners the ability to effectively warp to your town to hear you without leaving theirs.  

For locally-popular artists seeking even-wider audiences, at least, Every Place at Once is an experimental partial answer: an algorithmic semi-global explorer of the music distinctively popular in individual cities. That relies on a fair amount of listening signal to operate, though, and thus doesn't really answer the question about getting started. How do you get to be locally popular? How do you move from you playing your songs for friends to strangers listening to your songs of their own accord?  

I had been mostly ignoring this problem, having tried pushing the thresholds of Every Place at Once lower with results that were more worse than better, but periodically some new potential computational approach occurs to me. And usually also doesn't work. But this week, actually, one of these failed to fail as conclusively. It turns out that even at very low listening levels (on the order of tens of listeners, not even hundreds), if most of a song's listeners are in a single place, there's a pretty good chance that there's a reason for that. And, usefully, "That's where the band is from" turns out to be the most common one.  

So I made another thing. If artists with tens of fans are the scale where you might play house concerts, this thing is an attempt at algorithmic semi-global Hyperspace House Concerts.  

 

You can see pretty quickly that it's at least sometimes working: if you're listening to Harvard or MIT a cappella groups, you're probably in Cambridge. Mjangles is a rapper born in Ghana and raised in the Bronx, but he's currently a sophomore at Harvard. And even the music that isn't from Cambridge sometimes turns out to have interesting local stories. Jocelyn Hagen is a choral composer from North Dakota, but the week the list above was generated, her piece "Moon Goddess" was being rehearsed by a Harvard choir for an imminent concert. I didn't find as clear an explanation for André Caplet's similarly-lovely choral piece, but it was once performed by the Radcliffe Choral Society, so perhaps it still lingers in the walls.  

And if some of this signal turns out to be noise, maybe that's OK. Sometimes the music in a place is coming out of the open windows of a passing car. And if 10 cars pass you blasting the same song, now it's part of your city.  

So poke around, listen, see what you can find. Start where you live, and then try some places where you don't. There are lists for 500+ cities around the world, automatically updated every week, and more will appear as listening allows. And your listening can be part of it, part of how music travels and how careers begin and how we all find out what we're like.

On 27 April 2018 I gave a talk at the 2018 Pop Conference called "Panic, Death and Other Gender Patterns in Spotify Listening".  

The talk itself involved enough snapshots of changing data and short blasts of music that I am not going to attempt to transpose it directly to blog form, but the main point of it was to introduce some new things I have added to everynoise.com having to do with music and gender and numbers and genres and accountability and hope.  
 

Demographics and Listening  

The first of these things is an interactive index to the distinctive musical tastes of Spotify listeners divided by country/age/gender demographic groups. It is here:  

Every Demographic at Once  

This is similar in structure to Every Place at Once, which explores listening by city, and I retrofitted some new features from the demographic version to the geographic one, so both now try to describe each group's tastes in both genre and song terms.  

The demographic groups include, where data allows, country/age groups who self-identify as nonbinary, which Spotify began offering as a gender choice in September 2016.  

There's even a playlist of the music most distinctively popular among nonbinary listeners as a global group, which I played some songs from in the talk, and which you can hear here:  

 
 
 

Streamshare  

The second thing I discussed is an automatically-updating dataview breaking down the current share of Spotify streams in each country by artist-gender. That's here:  

Female-Artist Streamshare by Country  

It has data for each country overall, but also separately for "direct" listening (people going directly to album or artist pages and explicitly playing something), for Spotify's algorithmic personalized Discover Weekly playlists, and for Spotify's featured editorial playlists.  

The initial state of this data was about like this:  

- Per-country female-artist streamshare, by the most optimistic definition of "female artist" I could construct from my existing data, which is "music whose performers are not exclusively male", varies from about 16-33%.  

- You might hope, or even expect, that listeners might seek out more female artists on their own than when they are just listening to other people's playlists or editorial programming, but in fact it's currently the reverse, and female streamshare of direct listening is consistently lower.  

- Discover Weekly has notably lower female-artist streamshare than overall or direct listening. This is not very surprising, because DW is powered by co-reference in Spotify playlists, and globally male listeners make more and longer playlists than female listeners on average, so this is a pretty textbook example of algorithmic confirmation bias due to inherent asymmetries in the data inputs.  

- Spotify editorial programming, on the other hand, actually has a significantly higher percentage of female-artist streams in general and in almost every country. (Which is an intentional state brought about and maintained by specific Spotify editorial effort.)  
 

Genre Patterns  

The third thing is a similar automatically-updating dataview of genres and their shares of both female listeners and streams from female or mixed-gender artists.  

Gender Listening Patterns by Genre  

There is a wide array of interesting individual differences across the genre spectrum, but the discouragingly unsurprising overall insight is that male listeners stream less from female artists than female listeners do, and in general even female listeners don't stream equally.  

In an attempt to see what I could do to push against these inclinations, I took this genre and gender data and produced an experimental set of playlists that try to collect music by female or mixed-gender artists in every individual specific microgenre. The "genre" column in this table links to these, and initially about 1000 genres had enough artist-gender and genre-listening data to support playlists.  

 

Obviously this is not the first time anybody has tried to make female-artist genre playlists, but to me these efforts too often use only a very reductive high-level notion of genre, like "Women in Pop" and "Women in Rock", and thus seem inherently tokenistic. This also tends to implicitly characterize the presence of women in music as a novelty and a separate subject from music itself. The converse ideal, I think, is for it to be true and well-understood that there are women making every kind of music, and for it to be possible to find and listen to filtered subsets of genre music by female or mixed-gender artists without this seeming token or novel. At best these lists would be amazing just because music made by humans is consistently and profoundly amazing.  

In many genres, as you can trivially discover by exploring these lists, we are nowhere near this gender-balanced ideal. In some cases my robots couldn't find any plausible intersections between genre and gender at all, and in some of the playlists they did generate, the artists may be "mixed gender" on dubious technicalities, or the music may be more culturally adjacent to the genre than in it.  

But the good news about this set of battles in the fight to undo the miserable legacies of chauvinism and patriarchy is that at worst you'll hear some additional music by men, which is sometime surprisingly decent, or you'll end up listening to some music that isn't exactly what you asked for, which is also not a terrible idea for life.  
 
 

[The playlist I used for example snippets during the talk will make very little explanatory sense on its own, but it has music, and so I include it here for posterity, in case posterity runs out of music of its own.]  

 
 

[5 June 2018: For anybody who wants a concise tl;dr instead of these detailed breakdowns, I added a summary.]

*or the elders, or the girls, or the boys  

I added two more optional sort-orders to the list view of Every Noise at Once today.  

Youth sorts the genres by the average self-reported ages of each genre's artists' listeners. Thus the genres at the top are the ones listened to most uniformly by younger listeners, and the ones at the bottom are the ones mostly only old people like. (If you hover over the rank numbers on the left, you can see the actual average ages.)  

The youngest genre by this measure is Pixie, which is a hyper-poppy strain of pop-punk/-emo/-screamo, but "hyper-poppy pop-punk/-emo/-screamo" is ungainly, so I made up a name for it. I think it's a pretty good name, and I encourage you to work it into everyday conversation as if of course everybody calls it that.  

The oldest-listener genre, and one of only 2 genres whose average listener-age is older than I am, is Indorock, a bizarre 1950s repatriation of Dutch Indonesian colonialism back to the Netherlands after Indonesian independence. Probably this was the Pixie of its time and place, but that time was a really long time ago, and that fact that you can listen to it on a streaming music service in 2016 at all is fairly astonishing.  
 

Femininity sorts the genres by what percentage of each genre's self-identified male/female listeners self-identify as female. Spotify sign-up forms only offer three gender options at the moment ("female", "male" and just leaving it blank), so the current data is artificially binary, and thus the genres at the top are the ones with the highest ratio of female listeners to male, and the ones at the bottom are the most dominantly male-not-female.  

The most feminine genre by this measure is Teen Pop, which is rather stereotype-reinforcing, but the second one is the fanfic-pop genre Wrock, which I'm pretty sure you didn't expect, because statistically you probably didn't know that there's Wizard Rock to begin with, let alone that they call it "Wrock" for short, never mind that the Hermiones have more tolerance for it than the Harrys.  

The least-feminine genres at the bottom of the list are a roiling quagmire of auralized testosterone, the last 15 all explicitly involving death or brutality or brutal death or deathly brutality or grinding. I'm thinking I should really rename Djent to "Brutal Deathdjent Grind" just so it fits in better.  

As if the binary thing wasn't embarrassing enough, this data reveals that, at the moment, 72 genres skew more female than male, and 1363 skew more male than female. Only 9 genres have more than 60% female listeners, while 188 have more than 60% male listeners. Spotify's gender-self-identified listenership is about 53% male to start with, and small absolute differences can produce dramatic tipping effects, but that still doesn't seem to me like even vaguely a strong enough bias to account for this by itself.  

My first guilt theory, honestly, since it's mostly me that determines the genres in the genre-space, was that I over-model male-centric genre-areas, and thus the map presents a vastly unbalanced view of gender-balanced listening. To my superficial relief, at least, the basic gender disparity exists at the underlying artist level. Artists with more male listeners than female outnumber the reverse by about 4 to 1, and artists with more than 60% male listeners outnumber artists with more than 60% female listeners by almost 8 to 1. At the 90% threshold it's more than 40 to 1. Female listeners definitely gravitate towards a smaller set of core artists, and thus too a smaller set of genres.  

But do they "gravitate", as the result of innocent natural forces? Or are they pushed by some invisible forces generated by the ways in which music is made and distributed and presented? I don't know, and I feel like maybe somebody should try to find out, and I have a simultaneously sinking and inspiring feeling that maybe nobody is in a materially better position to find out than me.  
 

[A little further refinement from later: younger male listeners (<30) and older female listeners (30+) have mostly consistent shares of listening across the popularity spectrum. The big differences are between younger female listeners, who make up 40% of the audience for the most popular artists but only 20% for less popular artists, and older male listeners, who represent 13% for the most popular artists but 30% for less popular ones.]  

[PS2: A very cursory examination of the usage of Discover Weekly, Spotify's personalized weekly music-discovery playlist, seems consistent with all of the above: it represents a notably larger share of overall Spotify listening for older male listeners than for younger female ones. But, emphasizing the always-important point that individuals are not averages, among people who listen to their Discover Weekly lists actively, the age and gender differences essentially disappear. So maybe it's the idea of "discovery" itself whose appeal varies.]  

[PS3: The global disparity varies in magnitude across regions, but is present almost everywhere. The one major exception is Sweden, where the most popular artists do not skew towards either gender en masse. The effect is also fairly weak in the Netherlands, and tails off relatively quickly in Spain. But it is observable pretty much everywhere else, reaching an extreme in the Philippines, where the top 100 artists average 49% young female listeners but only 8% older male listeners.]

[This is the script from a talk I delivered at the EMP Pop Conference today. It was written to be read aloud at an intentionally headlong pace, with somewhat-carefully timed blasts of interstitial music. I've included playable clip-links for the songs here, but the clips are usually from the middles of the songs, and I was playing the beginnings of them in the talk, so it's different. The whole playlist is here, although playing it as a standalone thing would make no sense at all.]  

 

I used to take software jobs to be able to buy records, but buying records is now a way to hear all the world's music like collecting cars is a way to see more of the solar system.  

So now I work at Spotify as a zookeeper for playlist-making robots. Recommendation robots have existed for a while now, but people have mostly used them for shopping. Go find me things I might want to buy. "You bought a snorkel, maybe you'd like to buy these other snorkels?"  

But what streaming music makes possible, which online music stores did not, is actual programmed music experiences. Instead of trying to sell you more snorkels, these robots can take you out to swim around with the funny-looking fish.  

And as robots begin to craft your actual listening experience, it is reasonable, and maybe even morally imperative, to ask if a playlist robot can have an authorial voice, and, if so, what it is?  

The answer is: No. Robots have no taste, no agenda, no soul, no self. Moreover, there is no robot. I talk about robots because it's funny and gives you something you can picture, but that's not how anything really happens.  

How everything really happens is this: people listen to songs. Different people listen to different songs, and we count which ones, and then try to use computers to do math to find patterns in these numbers. That's what my job actually involves. I go to work, I sit down at my desk (except I actually stand at my fancy Spotify standing desk, because I heard that sitting will kill you and if you die you miss a lot of new releases), and I type computer programs that count the actions of human listeners and do math and produce lists of songs.  

So when anybody talks about a fight between machines and humans in music recommendation, you should know that those people do not know what the fuck they are talking about. Music recommendations are machines "versus" humans in the same way that omelets are spatulas "versus" eggs.  

So the good news is that you can stop worrying that robots are trying to poison your listening. But the bad news is that you can start worrying about food safety and whether the people operating your spatulas have the faintest idea what food is supposed to taste like.  

Because data makes some amazing things possible, but it also makes terrible, incoherent, counter-productive things possible. And I'm going to tell you about some of them.  

Counting is the most basic kind of math, and yet even just counting things usefully, in music streaming, is harder than you probably think. For example, this is the most streamed track by the most streamed artist on Spotify:  

Various Artists "Kelly Clarkson on Annie Lennox"  

Do you recognize the band? They are called "Various Artists", and that is their song "Kelly Clarkson on Annie Lennox", from their album Women in Music - 2015 Stories.  

But OK, that's obviously not what we meant. We just need to exclude short commentary tracks, and then this is the most streamed music track by the most streamed artist on Spotify:  

Various Artists "El Preso"  

Except that's "Various Artists" again. The most streamed music track by an actual artist on Spotify is:  

Rihanna "Work"  

OK, so that's starting to make some sense. Pretty much all exercises in programmatic music discovery begin with this: can you "discover" Rihanna?  

Spotify just launched in Indonesia, and I happen to know that Indonesian music is awesome, because there are people there and they make music, so let's find out what the most popular Indonesian song is.  

Justin Bieber "Love Yourself"  

I kind of wanted to know what the most popular Indonesian song is, not just the song that is most popular in Indonesia. But if I restrict my query to artists whose country of origin is Indonesia, I get this:  

Isyana Sarasvati "Kau Adalah"  

Which seems like it might be the Indonesian Lisa Loeb. It's by Isyana Sarasvati, and I looked her up, and she is Indonesian! She's 23, and her Wikipedia page discusses the scholarship she got from the government of Singapore to study music at an academy there, and lists her solo recitals.  

It turns out that our data about where artists are from is decent where we have it, but a lot of times we just don't. 34 of the top 100 songs in Indonesia are by artists for whom we don't have locations.  

But remember math? Math is cool. In addition to counting listeners in Indonesia, we can compare the listening in Indonesia to the listening in the rest of the world, and find the songs are that most distinctively popular in Indonesia. That gets us to this:  

TheOvertunes "Cinta Adalah"  

That is The Overtunes, who turn out to be a band of three Indonesian brothers who became famous when one of them won X Factor Indonesia in 2013.  

But that's still not really what I wanted. It's like being curious about Indonesian food and buying a bag of Indonesian supermarket-brand potato chips.  

I kind of wanted to hear some, I dunno, Indonesian Indie music. I assume they have some, because they have people, and they have X Factor, and that's bound piss some people off enough to start their own bands.  

So if we switch from just counting to doing a bit more data analysis -- actually, quite a lot of data analysis -- we can discover that yes, there is an indie scene in Indonesia, and we can computationally model which bands are more or less a part of it, and without ever stepping foot in Indonesia, we can produce an algorithmic introduction to The Sound of Indonesian Indie, and it begins with this:  

Sheila on 7 "Dan..."  

That is Shelia on 7 doing "Dan...", and I looked them up, too. Rolling Stone Indonesia said that their debut album was one of the 150 Greatest Indonesian Albums of All Time, and they are the first band to sell more than 1m copies of each of their first 3 albums in Indonesia alone.  

Of course, they're also on Sony Music Indonesia, and I assume that at least some of those millions of people who bought their first 3 albums, before Spotify launched in Indonesia and destroyed the album-sales market, are still alive and still remember them. One of the hard parts about running a global music service from your headquarters in Stockholm and your music-intelligence outpost in Boston, is that you need to be able to find Indonesian music that people who already know about Indonesian music don't already know about.  

But once you've modeled the locally-unsurprising canonical core of Indonesian Indie music, you can use that to find people who spend unusually large blocks of their listening time listening to canonical Indonesian Indie music (most of whom are in Indonesia, but they don't have to be; some of them might be off at a music academy in Singapore, where Spotify has been available since 2013), and then you can calculate what music is most distinctively popular among serious Indonesian Indie fans, even if you have no data to tell you where it comes from. And that gets us things like this:  

Sisitipsi "Alkohol"  

That is "Alkohol" by Sistipsi. A Google search for that song finds only 8400 results, which appear to all be in Indonesian. Their band home page is a wordpress.com site, and they had 263 global Spotify listeners last month.  

PILOTZ "Memang Aku"  

PILOTZ, with a Z. Also from Indonesia! 117 listeners.  

Hellcrust "Janji Api"  

Hellcrust. 44 listeners last month. I looked them up, and yes, they're from Jakarta.  

199x "Goodest Riddance"  

199x. 14 monthly listeners! Also, maybe actually from Malaysia, not Indonesia, but in music recommending it's almost as impressive if you can be a little bit wrong as it is if you can be right, because usually when you're wrong you'll get Polish folk-techno or metalcore with Harry Potter fanfic lyrics.  

So that's what a lot of my days are like. Pose a question, write some code, find some songs, and then try to figure out whether those songs are even vaguely answering the question or not.  

And if the question is about Indonesia, that method kind of works.  

But we also have 100 million listeners on Spotify, and we would like to be able to produce personalized listening experiences for each of them. Actually, we'd like to be able to produce multiple listening experiences for each of them. And we can't hire all of our listeners to work for us full-time curating their own individual personal music experiences, because apparently the business model doesn't work? So it's computers or nothing.  

People, it turns out, are somewhat harder than countries.  

For starters, here is the track I have played the most on Spotify:  

Jewel "Twinkle, Twinkle Little Star"  

As humans, we might guess that it is not quite accurate to say that that is my favorite song, and we might have a very specific theory about why that is. As humans, we might guess that the number of times I have played the song after that has a different meaning:  

CHVRCHES "Leave a Trace"  

In the latter case, I love CHVRCHES so much. But in the former case, I love my daughter even more than I love CHVRCHES, and some nights she really needs to hear Jewel sing "Twinkle Twinkle Little Star" at bedtime.  

And if we are still in the early days of algorithmically programmed listening experiences, at all, then we're in what I hope we will look back on as the early- to mid- prehistory of algorithmic personalized listening experiences. I can't tell you exactly how they work, because we're still trying to make them work. But I can tell you 7 things I've learned that I think are principles to guide us towards a future in which dumbfoundingly amazing music you could never find on your own just flows out of the invisible sea of information directly into your ears. When you want it to, I don't mean you can't shut it off.  

1. No music listener is ever only one thing.  

I mean, you can't assume they are. I have a coworker named Matt who basically only listens to skate-punk music, ever, and we test all personalization things on him first, because you can tell immediately if it's wrong. Right: Warzone "Rebels Til We Die". Wrong: The Damned "Wounded Wolf - Original Mix". But other than him, almost everybody turns out to have some non-obvious combination of tastes. I listen to beepy electronica (Red Cell "Vial of Dreams") and gentle soothing Dark Funeral "Where Shadows Forever Reign" and Kangding Ray "Ardent", and sentimental Southern European arena pop (Gianluca Corrao "Amanti d'estate"), and if you just average that all together it turns out you mostly end up with mopey indie music that I don't like at all: Wyvern Lingo "Beast at the Door"  

2. All information is partial.  

We know what you play on Spotify, but we don't know what you listen to on the radio in the car, or what your spouse plays in your house while you're making dinner, or what you loved as a kid or even what you played incessantly on Rdio before it went bankrupt. For example, this is one of my favorite new albums this year: Magnum "Sacred Blood 'Divine' Lies". I adore Magnum, but I hadn't played them on Spotify at all. But my robot knew they were similar to other things it knew I liked. Sometimes music "discovery" is not about discovering things that you don't know, it's about the computer inferring aspects of your taste that you had previously hidden from it.  

3. Variety is good.  

It is in the interest of listeners and Spotify and music makers if people listen to more and more varied music. If all anybody wanted to hear was this once a day -- Adele "Hello" -- there would be no music business and no streaming and no joy or sunlight. Part of my job is to crack open the shell of the sky. Terabrite "Hello". If you are excited to hear what happens next, you will be more likely to pay us $10, and we will pay the artists more for the music you play, and they will make more of it instead of getting terrible day-jobs working for inbound marketing companies, and the world will be a better place.  

4. People like discovering new music.  

They may hate the song you want them to love. They may have a limited tolerance for doing work to discover music, or for trial-and-erroring through lots of music they don't like in order to find it, but neither of those things mean that they wouldn't be thrilled by the right new song if somebody could find it for them. One of you will come up after this to ask me what this song is: Sweden "Stocholm". One of you, probably a different person, will wonder about this: Draper/Prides "Break Over You". I have like a million of those. I mean actually like an actual million of those.  

5. Bernie Sanders is right.  

It is in the interest of the world of music creators if the streaming music business exerts a bit of democratic-socialist pressure against income inequality. The incremental human value of another person listening to "Shake It Off" again is arguably positive, but it's probably also considerably smaller than the value of that person listening to a new song by a new songwriter who doesn't already have enough money to live out the rest of their life inside a Manhattan loft whose walls are covered with thumbdrives full of bitcoins and #1-fan selfies. Anthem Lights "Shake It Off". Taylor, if you're listening, I'm going to keep playing shitty covers of your songs until you put the real ones back on Spotify. That's how it works.  

6. If you're going to try to play people what they actually like, you have to be prepared for whatever that is.  

DJ Loppetiss "Janteloven 2016"  

That's "Russelåter", which is a crazy Norwegian thing where high school kids finish their exams way before the end of the senior year, so in the spring they get together in little gangs, give themselves goofy gang names, purchase actual tour buses from the previous year's gangs, have them repainted with their gang logo, commission terrible crap-EDM gang theme songs from Norwegian producers for whom this is the most profitable local music market, and then spend weeks driving around the suburbs of Oslo in these buses, drinking and never changing their clothes and blasting their appalling theme songs. I did not make this up.  

7. Recommendation incurs responsibility.  

If people are going to give up minutes of their finite lives to listen to something they would otherwise never have been burdened with, it better have the potential, however vague or elusive, to change their life. You can't, however tantalizing the prospect might seem, just play something because you want to. (Aedliga "Futility Has Its Limits") Like I said, you definitely can't do that. If you do that, the robots win.  

Thank you.

Thanks to a couple people's puzzled questions, I just realized that I wrote a pretty great bug a couple days ago in my code that makes the main genre map for Every Noise at Once. It needs to play one example song for each genre, so when my other code that makes a whole playlist for each genre runs, it copies the lead-off song from each playlist into another file for use by the main map. The genre playlists have logic that tries very hard to put songs in order by how well they represent the genre, so the first track is hopefully the one with the best combination of cultural and acoustic relevance. As you go down the list, you get further from the center.  

The line that saved a genre track for use by the main map was inside the loop of code that picked all the genre-playlist tracks. It correctly saved just a single track, so that was good. But it did so every time the playlist code picked any track, not just the first time. So it saved the best track, and then it saved the second-best track over the best track, and it kept doing that until it had saved the worst track on the playlist as the "best" example! And then it smugly stopped.  

It is an oblique and kind of impressive testimony to how well the whole process works that most of the worst examples were basically still plausible. And the bug prompted me to look more closely at the cutoff criteria, and tweak things so that sufficiently dubious tracks towards the end of the genre playlists aren't included at all.  

But as perverse results of small-seeming errors go, that was pretty impressive.

Along with adding a few more genres to Every Noise at Once, I've also just added two more levels of listening depth. So most genres now have three Spotify playlists instead of just one:  

The Sound of...: This is the existing one, which is an attempt at a data-generated algorithmic faintly-canonical introduction to that music. If you don't know the genre that well, this is where to start.  

The Pulse of...: This one uses our core data about the genre to find the listeners who know it best, and then uses the distinctive listening patterns of those listeners to find the genre's current heavy-rotation. This can be a mix of new and old, well-known and obscure, core and fringe. The math is merely organizing the collective will of the fans. For genres you already know well, this one might be more intriguing.  

The Edge of...: This one uses the same approach as The Pulse, but attempts to restrict the results to new and mostly unknown music that the genre's fans have discovered. This is the dangerous frontier, where your safety cannot be guaranteed. Explore with curiosity, and don't be afraid to keep a hand near the Next button.  

There are links to all of these at the top of each genre page. They're also linked in the description of each The Sound Of list.  

 

(There are a few genres that don't have enough new-and-unknown music to produce a meaningful The Edge, and a few that don't have enough listeners to even get a Pulse. But fewer than you might think. It's a big world, and even things you've never heard of usually turn out to be somebody's whole scene.)

For convoluted logistical reasons, the maps on Every Noise at Once haven't been updated in a while. The associated The Sound Of playlists have kept updating every week, but the genre list itself, the artist/genre assignments, and some of the other bits of map infrastructure have been involved in a slow belated technical migration as a result of the acquisition of the Echo Nest by Spotify two years ago.  

But this is all done now, or close enough, and the map is updated and updating again!  

Presumably nobody was really waiting impatiently to find out how many pixels left or right aggrotech should have shifted during this time. Encouragingly, ripping out all the data for the map and replacing it with new data didn't radically change everything. By which I mean, of course, that it initially wrought abject chaos, but after a day or two of frowning at code I wrote a long time ago I got the entropy basically re-contained. Mostly the products of this migration are straightforwardly good, and just magically make everything better. But sometimes the magic fails, and it takes a little human attention to allow the betterness to assert itself properly, so if you notice anything strange, let me know.  

More notably, reactivating the whole system makes it possible to add and remove genres again. A few marginal genres basically got absorbed by the data-fueled expansion of more prominent or interesting things, but mostly the deeper data made it possible to model known things I hadn't been able to include before, and to give names to obscure and/or emerging clusters that our old instruments weren't powerful enough to clearly discern. The net effect takes us to 1435 genres as of today, and these are the new ones:  

african gospel
alt-indie rock
anthem emo
anthem worship
bass trap
beats
chamber choir
chamber psych
channel pop
christian relaxative
cornetas y tambores
czech hip hop
danspunk
deep australian indie
deep big room
deep cumbia sonidera
deep danish pop
deep german pop rock
deep groove house
deep indie r&b
deep latin hip hop
deep melodic euro house
deep new americana
deep pop edm
deep pop r&b
deep southern trap
deep swedish indie pop
deep taiwanese pop
deep underground hip hop
drill
electro bailando
finnish dance pop
fluxwork
francoton
french indietronica
german street punk
groove room
hungarian hip hop
indie anthem-folk
indiecoustica
indie garage rock
indie poptimism
indie psych-rock
indie rockism
kids dance party
kwaito house
lift kit
norwegian indie
otacore
pixie
pop flamenco
pop reggaeton
post-screamo
preverb
redneck
romantico
russelater
slow game
spanish noise pop
spanish rock
strut
swedish eurodance
swedish idol pop
teen pop
tracestep
vapor pop
vapor soul
vapor twitch
voidgaze
west coast trap  

Some of these, to be clear, although not necessarily the ones you think, I made up. That is, they are names I made up for music and listening modes that I did not make up. Whether this makes them "really" genres is an existential question we can debate while we listen.