You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for creating this class. It's very useful, but I've ran into one error: Currently isLessThan can return the wrong answer if the exponents differ by one.
Steps to reproduce:
var m9e2 = Big.new(9, 2)
var m01e3 = Big.new(0.1, 3)
print(m01e3.isGreaterThan(m9e2))
print(m9e2.isGreaterThan(m01e3))
print(m9e2.isLessThan(m01e3))
print(m01e3.isLessThan(m9e2))
returns:
true
false
true
false
isLessThan simply returns true if exponent < n.exponent and always returns false if exponent > n.exponent.
Possible fix:
## Equivalent of [code]Big < n[/code]
func isLessThan(n) -> bool:
n = Big._typeCheck(n)
Big.normalize(n)
if (mantissa == 0
and (n.mantissa > MANTISSA_PRECISION or mantissa < MANTISSA_PRECISION)
and n.mantissa == 0
):
return false
if exponent < n.exponent:
if exponent == n.exponent - 1 and mantissa > 10*n.mantissa:
return false
return true
elif exponent == n.exponent:
if mantissa < n.mantissa:
return true
return false
else:
if exponent == n.exponent + 1 and mantissa * 10 < n.mantissa:
return true
return false
The text was updated successfully, but these errors were encountered:
I can reliably recreate this issue as well. I have used the code provided to success. @CyranosaurusRekt you should create a pull request with your suggested fix.
It looks like negative numbers don't get accounted for. So if the big number is m -5.123 e 3 and you are checking if it's less than 0 or any positive number less than 5123 it will return false.
I think the solution is that we should just manually verify negative numbers.
func isLessThan(n) -> bool:
n = Big._typeCheck(n)
Big.normalize(n)
if (mantissa == 0
and (n.mantissa > MANTISSA_PRECISION or mantissa < MANTISSA_PRECISION)
and n.mantissa == 0
):
return false
if exponent < n.exponent:
if exponent == n.exponent - 1 and mantissa > 10*n.mantissa:
return false #9*10^3 > 0.1*10^4
return true
elif exponent == n.exponent:
if mantissa < n.mantissa:
return true
return false
else:
if exponent == n.exponent + 1 and mantissa * 10 < n.mantissa:
return true
# if it's negative and the other is positive ofcourse it's less regardless of exponent
if mantissa < 0 and n.mantissa >= 0:
return true
return false
Thanks for creating this class. It's very useful, but I've ran into one error: Currently isLessThan can return the wrong answer if the exponents differ by one.
Steps to reproduce:
returns:
isLessThan simply returns true if
exponent < n.exponent
and always returns false ifexponent > n.exponent
.Possible fix:
The text was updated successfully, but these errors were encountered: