Yes a FTDI USB to serial is exactly what you need along with esptools or pyflasher plus the firmware.bin file.
The latest version available for the ESP8266 WiFi is 2.8.2 available here:
The instructions for reload are here:
I just did an update of the Wifi of my EVSE kit from 2.7.5 to the most recent version. The update went successfully (made sure I didn't press the upload button twice as that appeared to cause issues), but directly after the upload, the website was unresponsive and a restart didn't make things better either.
I've been reading I can flash the wifi module manually. Can I use any ftdi usb to serial converter for that purpose? I should have one at hand (couldn't find it yet, though) so I rather not buy one if not needed.