The reason for bespoke memory cards on consoles wasn't because the flash was expensive, it was because it was expensive to licence the interface tech, e.g. compact flash would have been fine, but it would have cost significantly more. Sony let anybody make their own memory cards for the first two home PlayStation consoles.
MemoryStick launched in 1998 and was used on a lot of digital audio devices, including kit made by Aiwa, Kenwood, Pioneer, Sanyo and Sharp. It was smaller than CompactFlash and faster than SD card which didn't launch until 2000, and later became the most common format at the time, so it made a lot of sense in a portable performance device.
Of course they did. Sony were, and still are, one of the biggest manufacturers of professional A/V equipment in the world. Much of Hollywood relies on Sony cameras, reference monitors and digital formats. If you wonder why Sony have made so much media stuff it's principally been to solve problems in the professional production space, that has then filtered down into the consumer channel.
DAT was VHS replacing analogue with digital signals. Sony's various audio formats like LDAC were create to solve problems like the lack of a streaming container format in MP3 at the time. MiniDisc was supposed to replace tapes in portable players when CDs were too large, UMD was intended to provide a lot of cheap disc storage in something that could easily slot into a portable device with some case protection to minimise scratches to the disc.
Don't forgot the 3.5" floppy disk, also by Sony. Smaller than the 5.25" and 8" floppy discs and higher capacity. Every format and standard has been developed to solve a problem for which there wasn't an existing solution. They could have gone further, but Sony quickly embraced various USB and other I/O standards, they didn't care what what 3.5" and 2.5" drives you put into PS2, PS3 and PS4. The 60gb launch PS3 had a flash card reader. ¯\_(ツ)_/¯