From the example above, I would be putting all artists' albums in one jumbled table instead of individual artists' albums tables such as "Duran Duran's albums" or "Sting's albums".
That's true, but you get some flexibility by doing that. To start with, you now have a single, uniform representation of all of the data. You know that every album is in the album table, for example. You also get a query mechanism that lets you perform ad hoc queries you don't have to plan in advance, as long as your data fits the relational model. For example, to get the titles of all albums released in the 90s, you could write:
SELECT title FROM album WHERE release_year >= 1990 AND release_year <= 1999;
Or just Sting albums from the '90s:
SELECT title FROM album JOIN artist USING(album_id) WHERE release_year >= 1990 AND release_year <= 1999 AND artist.name = 'Sting';
Or Sting and the Police albums:
SELECT title FROM album JOIN artist USING(album_id) WHERE artist.name = 'Sting' OR artist.name = 'The Police';