How should I deal with big numbers / complex mathematics? - Bome Forums Archive 2005

andymees

2015-11-02 13:57:39

Hey All

Very new to all of this, and trying to build accurate translators for an old set of flying faders for use with Adobe Premiere.

The translator's transform function for mapping hardware fader control to the software faders is working okay.
All the possible values for variables calculated within the translator's rules fall within allowable range (i.e. within signed 32-bit integer range)

But the inverse transform function, for the translator which maps software fader control back to the hardware faders, is not working.
Lots of potential variable values that are well outside acceptable range + a constant value that is larger than acceptable range.

Is there a proper method / best practice for dealing with 32-bit integer limitations?

My translator should provide accurate translation using the transform below:

xx=oo+7756
ss=xx*xx // creates large values for ss, but within 32-bit integer range
pp=ss*241238 // can create values outside acceptable 32-bit integer range
rr=xx*475309458 // can create values outside acceptable 32-bit integer range
pp=pp-rr
pp=pp/306987237600 // this value is too large to be held and is truncated to 2044559584

where oo is the input value, ranging from -7756 to 5980

I've created an "approximate" transform that keeps values within range, but it's not accurate and the software and hardware ends up fighting itself with the faders readjusting after I've let them go.

Appreciate any help and guidance.
Best
Andy

florian

2015-11-03 01:11:44

Hi Andy,
very interesting problem! I'm afraid there is no straight forward solution. It should be possible to implement virtual 64-bit math by combining two 32-bit integers variables to form one 64-bit variable. But that is quite complicated...

Simplifying should be possible.

1) Simplifying the formula. The original formula is:
pp = ((xx * xx * 241238) - (xx * 475309458)) / 306987237600
This can be changed to:
pp = (xx * ((xx * 241238) - 475309458)) / 306987237600
This should reduce the range of intermediate results.

2) Reducing resolution
I haven't tried to find a common denominator for the three numbers. One "radical" way is to just eliminate by 241238:
pp = (xx * (xx - 1970)) / 1272549
I have checked in a spreadsheet, there are exactly 23 input values where the output value is off by one. I experimented a bit with the eliminator and found that reducing it slightly will offset the rounding errors a bit. Using 241231 instead of 241238 yields this formula:
pp = (xx * (xx - 1970)) / 1272586
Here, only 8 input values (of 13736) cause an output value off by one. If needed, you could handle these 8 cases in separate IF statements.

In any case, I believe that the intermediate results won't exceed the 32-bit signed int range.

Now the Rules can be written as:

Code: Select all

xx=oo+7756
pp=xx-1970
pp=pp*xx
pp=pp/1272586

Unfortunately, the last input value is one of the 8 wrong output values. So you may want to add at least this one:

Code: Select all

if xx=13736 then pp=127

Anyway, the spreadsheet is attached (OpenOffice/LibreOffice format).
Let me know if this helps!
Florian

Attachments

BomeForum_Topic11397_HighRes.ods: (300.81 KiB) Downloaded 147 times