Both the local and remote X11 applications are actually using the same X11 server: the one that is actually connected to the display you're physically viewing.
If the scaling of local X11 applications is OK, but remote X11 applications have messed-up scaling, then there must be some settings - possibly environment variables - that are affecting the scaling of local applications, but are not replicated to the remote session.
Assuming that the remote X11 applications use the same X11 toolkit libraries as the local ones with OK scaling, you'll just need to identify and replicate the missing settings on the remote session, and that might be the quickest way to fix the problem. (However, this might be just a workaround rather than an actual fix - see below)
If the remote X11 applications use a different toolkit, you might have to find the documentation of the remote toolkit library and find out which settings that toolkit requires for an equivalent effect.
However, you should also use
xdpyinfo | grep -e dimensions -e resolution and verify that the reported information makes sense - not only the pixel resolution, but also the Dots-Per-Inch values and the physical display size (reported in millimeters). These parameters normally come from the display's EDID information, which might sometimes be incorrect, and will throw off all X11 scaling unless worked around at the X11 toolkit level.
If you find that the physical display size and/or the DPI values don't make sense for your actual display, you should modify your local X server configuration to override the wrong values with correct ones. After applying the override and restarting the X11 server, you might have to undo any X11 toolkit-level workarounds for incorrect scaling, as they should no longer be needed and might now result in an error in the opposite sense, as the root cause of the scaling error will now be corrected.