# FAQ 4.2 Why is int() broken? - Page 2

•  Subject
• Author
• Posted on

## Re: FAQ 4.2 Why is int() broken?

On Mon, 04 Aug 2008 15:32:41 +0200, Peter J. Holzer wrote:

Well, not mass market, but gaining in popularity pretty fast. You won't
have one at home presumably, but all gigs I worked at the past ten years
had several of those. Yes several, running multiple virtual machines.

OTOH, those virtual machines typically ran AIX (or linux) and did not
take advantage (AFAIK) of the decimal floating point at all.

M4

## Re: FAQ 4.2 Why is int() broken?

Martijn Lievaart wrote:

But the decimal floating point feature is much less than ten years old,
so it's natural that it's not going to get much use until languages
start to be updated to support it. It's going to take a while in typed
languages, and untyped languages like Perl and Ruby are going to need
some philosophical examination. PL/I is about the only language that was
prepared.
--
John W. Kennedy
"The whole modern world has divided itself into Conservatives and
Progressives. The business of Progressives is to go on making mistakes.
The business of the Conservatives is to prevent the mistakes from being
corrected."
-- G. K. Chesterton

## Re: FAQ 4.2 Why is int() broken?

Peter J. Holzer wrote:

I probably should of been more clear. Yes, in the case of calulator and
pencil paper, it's about doing calculations with decimal (or Real)
numbers.

A TI-86 also uses 12 decimal places. But it depens how you enter things:

1/3
.333333333333
.333333333333 * 3
.999999999999

But:

123456789012345678901
1/3
.333333333333
Ans*3
1

The 2nd example uses the build in "Ans" variable which always contains
the result of the last computation, which can be more accurate than
simply re-typing what was displayed.

what happens if you enter "(1/3) * 3" instead of "0.333333333333 * 3" on

Just like with mine, theres a difference between what is known
internally and what is displayed.

On a TI-86, "1/3*3-1" gives me -1E14, but on a TI-89 it's 0. I have no
idea why the result is negative on the 86, unless the sign bit is
somehow being clobbered.

Mathetically, the TI-89 seems to get it more right, as (1/3)*3 = 1 is
mathematically correct, as the 3 in the demoninator in of the first
number and 3 in the numinator of the 2nd number (3/1) cancel each other
out, leaving 1/1 which is better writen as just plain ol' 1. Then 1-1
is, of course, 0.

I'm not sure why the TI-86 has such a strange result when "-1" is thrown
in, but the TI-89 doesn't have this problem, and it seems neither does
Perl:

\$ perl -e 'print +(1/3), "\n";'
0.333333333333333333

\$ perl -e 'print +(0.333333333333333333 * 3), "\n";'
0.999999999999999999

\$ perl -e 'print +(1/3*3), "\n";'
1

\$ perl -e 'print +(1/3*3-1), "\n";'
0

I think you might be right. I pulled the box that the contains the
motherboard and other parts from it and I can't tell just by looking it
seems. There is a big square chip I certainally remember quite well: the
one with "486 DX2" nicely written upon it. It was a very powerful
computer for it's time and one I have very fond memories of in the PC
glory days :-)

I assume s/387/386/ and you are of course right.

--
szr

## Re: FAQ 4.2 Why is int() broken?

szr wrote:

[...]

Oh wait, 387 was the math co-processor.

--
szr

## Re: FAQ 4.2 Why is int() broken?

I didn't enter 0.333333333333 * 3. I entered "1 ENTER 3 / 3 *". The
HP-48 uses Reverse Polish Notation. It also displays all digits (unless
told to display a rounded result), so that wouldn't make a difference
anyway.

Because 0.99999999999999 - 1  is obviously less than 0? My mistake - I
omitted the sign. From the result it is rather clear that the TI-86 uses
14 digits internally.

Nope. I clobbered the sign bit while typing. The result must obviously
be negative.

Since the TI-89 can do symbolic algebra it may actually notice this
never compute 1/3.

I have been trying to explain it to you in just about every posting in

hp

## Re: FAQ 4.2 Why is int() broken?

Peter J. Holzer wrote:

I hated those type of calculators. I think the difference is calculators
like the TI 86/89 seems to have a threshold for determining if a number
repeats ad infinitum. On the 86, if I type 0.333... with 12 decimal
digits (12 "3"'s after the ".") and multiply by 3, it's ".999999999999",
but if I add a 13th "3" and multiple by 3 it comes out as "1". It seems
calculators like yours do not do this. The TI-89 seems to be far better
with this, though.

Yeah, if it's storing the value internally as 0.99999999... than
subtracting 1 would yield a negative value. It's still a result one
might find surprising at first, and again, this doesn't seem to be a
problem on the TI-89.

Ah, ok.

Either that or it uses a threshold algorithm like I described (either
way the 89 seems to handle this far better than the 86.)

This was from only one reply up, so how could it have been from every
posting? It just didn't occur to me that the 86 was internally had a
value like 0.99999... which is smaller than zero. That's what tripped me
up.

--
szr

## Re: FAQ 4.2 Why is int() broken?

[...]

From what you wrote earlier I concluded that the TI-86 uses 14 decimal
digits internally, and displays only 12 (which is similar to the TI
calculators of 25 years ago, which used 11 digits internally and
displayed 8). So for the calculator 0.333333333333 and 0.3333333333333
are different numbers for the calculator, even if both are displayed as
0.333333333333. Multiply both by 3 and you get 0.999999999999 and
0.9999999999999 respectively. If the calculator displays 12 digits, it
can display 0.999999999999 exactly, but it needs to round
0.9999999999999: Since the first digit to be omitted is "9" it needs to
be rounded up, so the rounded value is 1.0. But the value which is
stored is still the correct value 0.9999999999999.

Right. The HP-48 displays the full result by default. It doesn't round
unless told.

As I said, the TI-89 does symbolic algebra. But you could try:

1/3

Ans*3

Ans-1

(force to numeric representation after each step if the calculator
doesn't do it automatically)

And check the result.

I don't think there's a threshold algorithm. Your example didn't show
one and it wouldn't work anyway.

Because the whole thread is about the fact that any floating point
representation has a fixed number of digits in a given base and can
therefore represent only a finite number of different values. It doesn't
matter whether you use base 2 or base 10 or base 42, whether you use 10
or 12 or 53 digits: Almost every result of a division must be rounded,
and therefore a/b*b is generally *not* a.

A decimal representation only has the psychological advantage of being
closer to what people expect.

It couldn't be anything else.

hp

## Re: FAQ 4.2 Why is int() broken?

Peter J. Holzer wrote:

Well, they are different numbers simply by having a different number of
digits.

Yeah, I think that's pretty much what it does.

That can in fact be a good thing in some situations.

On the TI-89:

_________________________
1/3                   1/3          Typed as: 1/3
1/3*3                   1          Typed as: ans(1)*3
-1                     -1          Typed as: ans(1)-1
_________________________

But, typing 1/3 it in with decimal points:

_________________________
1.
---         .333333333333          Typed as: 1./3.
3.
.333333333333*3        1.          Typed as: ans(1)*3
.999999999999 - 1                  Typed as: ans(1)-1
-1.E-14
_________________________

It goes to numeric when ever a decimal is thrown in. The 2nd example
works if you use "1/3+0.0", "1/3*1.0", etc, isntead of "1./3.". When no
decimal point is included around 1/3, then yeah it does seem to use
worlds.

See above examples :-)

By "threshold", I meant the threshold where a calculator considers a
repetitive number to be infinitely-repetitive and act accordingly:

On a TI-86:

_____________________
.333333333333*3
.999999999999
.3333333333333*3
1
_____________________

I've seen such threshold algorithms used in math libs in the past. Hell,
I remember once writing a little bit of code for fun that did that sort
of thing for a dec2frac function of a Fraction class in C++. I'm going
to see if I can dig it up.

Understood.

Yeah, which is usually what they did on pencil and paper. Any one doing
that math that way would expect, for instance, that a/b*b == a, but
you're right that in binary this is not always the case because it
cannot be represented exactly.

That came out wrong. I meant 0.9999999... internally *while* displaying
1 (1 is greator than 0 while 0.99999... is not - that was the source of
the confusion before.)

--
szr

## Re: FAQ 4.2 Why is int() broken?

[...]

It doesn't do this.

Ok. Now I really give up. Just a few lines above you seemed to accept my
explanation that this is simple rounding. Now you give the the same
example again ...

EOD.

hp

## Re: FAQ 4.2 Why is int() broken?

Peter J. Holzer wrote:

I think you misunderstood what I was saying and you over snipped and
lost some of the context. I understand what you said about rounding when
it comes to binary FP math, but it doesn't seem to be the case with the
TI-86 and you asked what I meant by "threshold and I simple answered.
Why are you so quick to assume this is not the case and that you are
automatically correct?

--
szr

## Re: FAQ 4.2 Why is int() broken?

Why do you think that this doesn't seem to be the case?

Because

a) it is the most simple explanation.

b) the evidence you produced matches perfectly

c) you haven't provides even a glint of an idea how it could be
different, much less any evidence that it really is different.

hp

## Re: FAQ 4.2 Why is int() broken?

Yes. From "Customize your HP-28", by W.A.C. Mier-Jedrzejowicz, page 57:

A real number consists of a 5-nybble address followed by the
number's 3-digit exponent and 12-digit mantissa, stored back to
front. Thus the number
-7.46454443424E105

is stored as:
3 3 9 2 0  5 0 1  4 2 4 3 4 4 4 5 4 6 4 7  9
\       /  \   /  \                     / \ /
9 if negative

[...]

Thanks, I'm busy enough as it is ;-).

hp

## Re: FAQ 4.2 Why is int() broken?

Obviously written by someone who doesn't know cobol at all well. Cobol has
been able to use binary since at least 1974. That 'normally' is completely
wrong, by default cobol wants to use binary. (at least since 1974 it has!)

Dave

--
Dave Stratford    ZFCA
http://daves.orpheusweb.co.uk /
Hexagon Systems Limited - Experts in VME systems development

## Re: FAQ 4.2 Why is int() broken?

Dave> Obviously written by someone who doesn't know cobol at all well. Cobol
Dave> has been able to use binary since at least 1974. That 'normally' is
Dave> completely wrong, by default cobol wants to use binary. (at least since
Dave> 1974 it has!)

But 1974 is "relatively modern" with respect to Cobol, right?

/me ducks

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

## Re: FAQ 4.2 Why is int() broken?

Randal L. Schwartz wrote:

Not really. COBOL has had at least two major version updates since, the
latest in 2002.

--
John W. Kennedy
Having switched to a Mac in disgust at Microsoft's combination of
incompetence and criminality.

## Re: FAQ 4.2 Why is int() broken?

Dave Stratford wrote:

The COBOL language did not distinguish binary from decimal until 1985.
Before that, most compilers for hardware that supported both bases
supplied some kind of language extension. In IBM, the tradition was to
use COMPUTATIONAL for binary and COMPUTATIONAL-3 for decimal
(COMPUTATIONAL-1 and COMPUTATIONAL-2 already being used for floating
point as a legacy from pre-360 systems).

In any case, all he said was "normally", which is true. Only an idiot
uses binary for dollars and cents if decimal is available to him.

--
John W. Kennedy
"Only an idiot fights a war on two fronts.  Only the heir to the
throne of the kingdom of idiots would fight a war on twelve fronts"
-- J. Michael Straczynski.  "Babylon 5", "Ceremonies of Light and Dark"

## Re: FAQ 4.2 Why is int() broken?

True. I only programmed in COBOL for about a year more than 20 years ago
(Microfocus Level II COBOL, IIRC).

I'm sure the compiler we used had a binary integer COMPUTATIONAL type,
maybe even a binary floating point type, but there was little reason to
use them in the type of application we were developing. In any case the
computational types felt like something tacked on as an afterthought - I
wouldn't have seen them as "default".

Right. You can, though - it just takes a little more thought.

hp

## Re: FAQ 4.2 Why is int() broken?

Jürgen Exner wrote:

I would not call the IBM 360/370/4300/390/zArchitecture series, nor the
IBM 3/3x/AS400/iSeries, nor, for that matter, the old 702/705/7080,
1401/1440/1460, 1410/7010, 650/707x, and 1620/1710 series, "not very
successful". For that matter, Intel has had support for decimal from the
beginning, though it's never been popular.

Moreover, IEEE-754r specifies decimal floating point, alongside the
binary floating point introduced by the original IEEE-754. IBM
zArchitecture, for one, already implements it.

"Rational numbers" is the usual term in English when speaking of
computer systems (normally software) that use arbitrary common fractions.

--
John W. Kennedy
"Only an idiot fights a war on two fronts.  Only the heir to the
throne of the kingdom of idiots would fight a war on twelve fronts"
-- J. Michael Straczynski.  "Babylon 5", "Ceremonies of Light and Dark"

## Re: FAQ 4.2 Why is int() broken?

^^^^^^^^^
^^^^^^^^^

He did not say that it required an infinite number of decimal digits.

He said that it required an infinite number of binary digits.

--