Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

isLessThan error when exponents differ by 1 #33

Open
CyranosaurusRekt opened this issue May 21, 2024 · 2 comments
Open

isLessThan error when exponents differ by 1 #33

CyranosaurusRekt opened this issue May 21, 2024 · 2 comments

Comments

@CyranosaurusRekt
Copy link
Contributor

Thanks for creating this class. It's very useful, but I've ran into one error: Currently isLessThan can return the wrong answer if the exponents differ by one.

Steps to reproduce:

var m9e2 = Big.new(9, 2)
var m01e3 = Big.new(0.1, 3)
print(m01e3.isGreaterThan(m9e2))
print(m9e2.isGreaterThan(m01e3))
print(m9e2.isLessThan(m01e3))
print(m01e3.isLessThan(m9e2))

returns:

true
false
true
false

isLessThan simply returns true if exponent < n.exponent and always returns false if exponent > n.exponent.

Possible fix:

## Equivalent of [code]Big < n[/code]
func isLessThan(n) -> bool:
	n = Big._typeCheck(n)
	Big.normalize(n)
	if (mantissa == 0
			and (n.mantissa > MANTISSA_PRECISION or mantissa < MANTISSA_PRECISION)
			and n.mantissa == 0
	):
		return false
	if exponent < n.exponent:
		if exponent == n.exponent - 1 and mantissa > 10*n.mantissa:
			return false
		return true
	elif exponent == n.exponent:
		if mantissa < n.mantissa:
			return true
		return false
	else:
		if exponent == n.exponent + 1 and mantissa * 10 < n.mantissa:
			return true
		return false
@floridaman
Copy link

floridaman commented May 23, 2024

I can reliably recreate this issue as well. I have used the code provided to success. @CyranosaurusRekt you should create a pull request with your suggested fix.

@nhold
Copy link

nhold commented Jun 18, 2024

It looks like negative numbers don't get accounted for. So if the big number is m -5.123 e 3 and you are checking if it's less than 0 or any positive number less than 5123 it will return false.

I think the solution is that we should just manually verify negative numbers.

func isLessThan(n) -> bool:
	n = Big._typeCheck(n)
	Big.normalize(n)
	if (mantissa == 0
			and (n.mantissa > MANTISSA_PRECISION or mantissa < MANTISSA_PRECISION)
			and n.mantissa == 0
	):
		return false
	if exponent < n.exponent:
		if exponent == n.exponent - 1 and mantissa > 10*n.mantissa:	
			return false #9*10^3 > 0.1*10^4
		return true
	elif exponent == n.exponent:
		if mantissa < n.mantissa:
			return true
		return false
	else:
		if exponent == n.exponent + 1 and mantissa * 10 < n.mantissa:
			return true
		# if it's negative and the other is positive ofcourse it's less regardless of exponent
		if mantissa < 0 and n.mantissa >= 0:
			return true
		return false

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants