It's actually pretty easy.  The opticsZoom values are in radians, and 
refer to the horizontal field of view measured from the center of the screen.  
If you have an optic with a 9 degree FOV, it should have opticsZoomMin = "9*pi/360" 
if you want it to fill a 4:3 screen horizontally.

9 * 3.14159265 / 360 = 0.0785398

That's the only important part.  Relative magnification depends on whatever you define as unity.  
I used ~0.3 radians.  ArmA by default uses 0.25 radians for rifle optics and 0.33 radians for tank optics.  
Angular magnification, of course, is just FOV at unity divided by the FOV of the optic.


If you want your optics model to actually match the FOV, you need a custom model.  
The BIS models aren't quite accurate enough.  
Mine are a 5.12x3.84 unit rectangle, of which the middle 1.00x0.75 units are visible on a 4:3 screen.  
More of the model is visible in widescreen resolutions.  
This is the exact size ratio used by the game--the actual size of the model doesn't matter.

Then, just define a certain number of pixels in each milliradian 
(if you've got a 1024x1024 texture and a 128 mil FOV, you've got 8 pixels per mil) and make the textures to that scale.

I actually worked everything the other way around, i.e. I 
defined a certain texture to be 14 px/mil and worked out the
d opticsZoom values based on that.  That made it a lot easier to make calibrated optics.