Qbasicnews.com
February 20, 2020, 10:23:21 AM *
Welcome, Guest. Please login or register.

Login with username, password and session length
News: Back to Qbasicnews.com | QB Online Help | FAQ | Chat | All Basic Code | QB Knowledge Base
 
   Home   Help Search Login Register  
Pages: [1]
  Print  
Author Topic: 1.#INF, how I hate thee  (Read 1938 times)
pr0gger
I hold this place together
*****
Posts: 775


« on: August 21, 2004, 03:57:35 PM »

I'm running into some problems with single precision variables in my program.  Every once in a while, the program will generate an error (Overflow), and I'll take a look at some of the variables and find that the problem arises whenever one (or more) of these values appear: when an operation is supposed to return zero... but one of these appears instead  (That's what's so frustrating-- it's not my fault, it's a result of binary math).

Is there any way to fix this?
Logged

size=9]"To announce that there must be no criticism of the president, or that we are to stand by the president, right or wrong, is not only unpatriotic and servile, but is morally treasonable to the American public." -- Theodore Roosevelt[/size]
Nexinarus
Wandering Guru
***
Posts: 301



WWW
« Reply #1 on: August 21, 2004, 05:38:07 PM »

hmm can you show some simple code of the problem you mean?
Logged
Moneo
Na_th_an
*****
Posts: 1971


« Reply #2 on: August 21, 2004, 09:45:04 PM »

Progger,

Do you have to use single precision floating point?
Let's asume you need precision to 2 decimal places, then:
* Use LONG variables.
* Use 3 IMPLIED decimal places, so at the end you can round to your required 2 decimal places.
* If the input to your program has 2 decimal places, multiply these numbers by 1000 to convert them to 3 implied decimal places while actually using a whole number as LONG.
* Do your calculations using the LONG numbers.
* At the end, divide you answer(s) by 100 while rounding, storing it into a floating point number, which will give you an answer with 2 decimal places.

I do this all the time to avoid all the crazy problems you get with floating point numbers.
*****
Logged
pr0gger
I hold this place together
*****
Posts: 775


« Reply #3 on: August 23, 2004, 11:45:03 AM »

Nex: I can't, actually... it's a very large program.
Moneo: I'm using some trig equations, so I try to be as precise as possible (i.e., 3-4 decimal places).    someone once told me that QB division is pretty slow, but this is really getting on my nerves so I'm gonna try that.

thanks
Logged

size=9]"To announce that there must be no criticism of the president, or that we are to stand by the president, right or wrong, is not only unpatriotic and servile, but is morally treasonable to the American public." -- Theodore Roosevelt[/size]
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Valid XHTML 1.0! Valid CSS!