Qbasicnews.com
February 26, 2020, 10:17:07 AM
 Pages: [1]
 Author Topic: Rounding numbers larger than 32k  (Read 3116 times)
Moneo
Na_th_an

Posts: 1971

 « on: January 15, 2005, 01:59:17 AM »

Rounding algorithms generally use the INT function which will not handle numbers greater than 32k.

CHALLENGE:
Write a subroutine, function or sub-program that can correctly round positive or negative numbers using the "conventional rounding method", which is: add .5 and truncate.

Input is a double precision number, which can be signed.
Output is also a double precision number,which is a whole number that can be signed.

*****
 Logged
Sterling Christensen
Na_th_an

Posts: 1328

 « Reply #1 on: January 15, 2005, 10:01:46 AM »

Code:
n# = 100000000.9#
n# = INT(n# + .5)
print n#

The result in this example is 100000001, which is too large for a LONG. Proof positive that INT returns a double, thus fitting all the requirements for the challenge . Yes, I'm a lazy cheater :lol:. No doubt there's a more interesting way to do it.
 Logged
Moneo
Na_th_an

Posts: 1971

 « Reply #2 on: January 15, 2005, 10:31:57 PM »

Sterling,

You're absolutely rignt. I tested it with up to 11 digit whole numbers with decimals, positive and negative, and it accepts them.

Where did I get the idea that INT could not handle numbers larger than the integer limit of 32k? I checked my manual, which is QuickBasic 4.0, although I'm running 4.5, and it says that it returns AN INTEGER. I even had underlined the word INTEGER. Maybe 4.0 had that restriction. Who knows?

In any event, SORRY GUYS FOR THE BOGUS CHALLENGE INFORMATION.
*****
 Logged
Z!re
*/-\*

Posts: 4599

 « Reply #3 on: January 16, 2005, 07:52:17 AM »

It does return an integer, but a mathematical integer. That is, a whole number.

Silly MS QB help.
 Logged
Moneo
Na_th_an

Posts: 1971

 « Reply #4 on: January 23, 2005, 10:41:32 PM »

I finally discovered where I saw the limit of 32k, It's in the CINT function, not the INT. I never use CINT 'cause it gives some weird results for certain negative numbers.
*****
 Logged
 Pages: [1]