This is Gilles' answer, saving it here so it doesn't get lost.
If you use the
sync mount option on the removable drive, all writes are written to the disk immediately, so you won't lose data from not-yet-written files. It's a bad idea, but it does what you're asking, kind of.
sync does not guarantee that you won't lose data. Unmounting a removable drive also ensures that no application has a file open. If you don't unmount before unplugging, you won't notice if you have unsaved data until it's too late. Unmounting while a file is open also increases the chance of corruption, both at the filesystem level (the OS may have queued some operations until the file is closed) and at the application level (e.g. if the application puts a lock file, it won't be removed).
sync is bad for the lifetime of the device. Without the
sync option, the kernel reorders writes and writes them in batches. With the
sync option, the kernel writes every sector in the order requested by the applications. On cheap flash media that doesn't reallocate sectors (meaning pretty much any older USB stick, I don't know if it's still true of recent ones), the repeated writes to the file allocation table on (V)FAT or to the journal on a typical modern filesystem can kill the stick pretty fast.
Therefore I do not recommend using the
sync mount option.
On FAT filesystems, you can use the
flush mount option. This is intermediate between
async (the default) and
sync: with the
flush option, the kernel flushes all writes as soon as the drive becomes idle, but it does not preserve the order of writes (so e.g. all writes to the FAT are merged).