4e changed a lot of things in an attempt to solve many of the problems with 3.5:

• Martials were (almost) as good as casters because they were given abilities that let them actually do things beyond saying “I full attack”
• Monster statblocks were simplified and divorced from the rules used to build PCs, instead just making sure that the numbers were right
• PCs were given more HP at low levels and the math was made less swingy so somebody didn’t get crit by a greataxe and insta-killed (something I personally have seen happen in the less than 20 sessions I’ve played of 3.5/PF)
• Modifiers were simplified to +2 for “minor” and +5 for “major” and IIRC generally didn’t stack, as a sort of proto-advantage system
• Spell slots were gone, and every class’ abilities were changed into one of 3 categories: Daily (recharges after an 8-hour long rest), Encounter (recharges after a 5-minute short rest), and At-Will (useable every turn)
• Additionally, non-combat spells/abilities were classed separately as “Utility Powers”, with the same cooldown system
• Classes were explicitly labeled with certain roles to give players an idea of what they were going to do (Strikers did damage, Controllers did CC (and damage), Leaders could heal and had support abilities, Defenders could generally lock down opponents with some sort of marking ability)
• The rules were written very technically, which stands at odds with 3.5e/PF and 5e, which are more naturalistic, resulting in fewer rules disputes

4e also had its share of problems, however, which turned many people off:

• The HP calculations were wrong in the first 2 monster manuals, resulting in long drawn-out slogs
• The amount of bookkeeping required to run combat increased by a fair bit, as now almost every PC and many NPCs could inflict status effects with varying duration
• Combat more or less required groups to use a grid (this was also largely true of 3.5, for what it’s worth)
• The Magic Item Treadmill was very much a thing, where every party member was expected to swap out almost all of their equipment every 5 levels to keep pace with the monster stats
• Healing was very gamified: you could only heal a certain number of times per day, regardless of how many potions you drank or spells were cast on you etc
• Character sheets were very complicated, and it was more or less required to use WotC’s software. It worked decently well and was easy to pirate, but it ran into another problem…
• The guy in charge of the software side of things at WotC murder-suicided his wife and the project was pretty much abandoned

The biggest issue facing 4e was that WotC’s business model during the 3e/3.5e era had been to publish dozens of books non-stop, and 4e was in no way backwards compatible. The licensing that had allowed 3rd party publishers to make stuff for 3.5e was generous enough for Paizo to make Pathfinder 1e, and an extremely large share of then-current players went to play PF instead. As for 5e’s popularity, it benefits from several things imo. First, it’s the current edition of the game, which means any new players will gravitate to it by default. Second is the effect of big-name actual plays like Critical Role, The Adventure Zone, or Dimension 20 which serve as excellent (if unrealistic) marketing. Lastly, nerd shit has gradually become more accepted in general, and so the audience for people who play D&D has increased.