__pairs and __ipairs metamethods have incorrect annotations

[chrisgbk]

How are you using the lua-language-server?

Visual Studio Code Extension (sumneko.lua)

Which OS are you using?

Windows

What is the issue affecting?

Annotations

Expected Behaviour

lua-language-server should specify correct function signatures for __pairs and __ipairs.

lua-language-server should in particular say that the __ipairs iterator generator function has 3 return values.

Following the annotation hints for __ipairs should not result in runtime errors.

Given that generic for k, v in pairs(t) --or ipairs(t) for pairs and ipairs are equivilant to:

do
  local f, s, var = pairs(t) --or ipairs(t)
  while true do
    local k, v = f(s, var)
    if k == nil then break end
    var = k
    <block>
  end
end

I believe these are the required function signatures:

__pairs = 
GEN_FUN(ITERABLE)
	RETURNS
		ITER_FUN(ITERABLE,KEY)
			RETURNS 
				KEY
				VALUE
		ITERABLE
		KEY_BEFORE_FIRST
OR nil

__ipairs = 
GEN_FUN(ITERABLE_ARRAY)
	RETURNS 
		ITER_FUN(ITERABLE_ARRAY,INDEX)
			RETURNS 
				INDEX OR nil
				VALUE
		ITERABLE_ARRAY
		INDEX_BEFORE_FIRST
OR nil

which I believe should be:

---@field __pairs (fun(t):((fun(t,k):(any,any)),any,any))|nil
---@field __ipairs (fun(t):((fun(t,k):((integer|nil),any)),any,integer))|nil

Actual Behaviour

__pairs annotation specifies the iterator function as taking a third argument, for the value. Lua versions 5.2 and 5.4 do not pass a third argument to the iterator function.

---@field __pairs (fun(t):((fun(t,k,v):any,any),any,any))|nil

__ipairs annotation specifies the iterator function as taking a third argument, for the value. Lua version 5.2 does not pass a third argument to the iterator function.

__ipairs annotation specifies the generator function as returning only 2 results. Following the annotation hinting results in nil being implicitly passed as the third argument, instead of the correct index.

---@field __ipairs (fun(t):(fun(t,k,v):(integer|nil),any))|nil

Reproduction steps

t = setmetatable({1,2,3}, {
  __ipairs = function(t) local function iter(t, k) k = k + 1 local v = t[k]; if v then return k, v end; end; return iter, t, 0 end,
})

for k, v in ipairs(t) do
  print(v)
end

(field) __ipairs: function|nil
function (t: any)
  -> fun(t: any, k: any, v: any):integer|nil
  2. any
function __ipairs(t: any)
  -> function
  2. unknown
  3. integer

Annotations specify that at most 2 return value(s) are required, found 3 returned here instead.Lua Diagnostics.(redundant-return-value)

Removing the extra return value as the annotation suggests results in :

.\sample.lua:2: attempt to perform arithmetic on local 'k' (a nil value)
stack traceback:
        .\source.txt:2: in function 'for iterator'
        .\source.txt:5: in main chunk
        [C]: in ?

Additional Notes

The example __ipairs sample is essentially what AI generated sample code looks like, so it is likely how someone would use __ipairs.

Lua 5.2 allows iteration with __ipairs to start at any point; this is useful if __ipairs is used to support non-standard ipairs iteration, such as starting from a negative index or starting at some arbitrary positive index, controlled by altering the third return value from the generator function to be a non-zero value.

Some of the __ipairs choices in the suggested annotations I made aren't technically explicitly mandated, but I feel they are in the spirit of the intended functionality; nothing stops anyone violating the implicit assumption that the keys are sequential, have numeric indexes, or they will be traversed in a given order:

t = setmetatable({a=1,b=2,c=3}, {
  __ipairs = function(t) local function iter(t, k) return next(t,k) end; return iter, t, nil end,
})

for k, v in ipairs(t) do
  print(k, v)
end

I kept my suggested annotation for __ipairs in line with the normal behaviors of ipairs when there is no __ipairs metamethod.

Log File

No response

[tomlau10]

I think the corresponding annotation are defined here:

lua-language-server/meta/template/basic.lua

Lines 248 to 253 in 7f76050

	---#if VERSION > 5.1 or VERSION == JIT then
	---@field __pairs (fun(t):((fun(t,k,v):any,any),any,any))|nil
	---#end
	---#if VERSION == JIT or VERSION == 5.2 then
	---@field __ipairs (fun(t):(fun(t,k,v):(integer|nil),any))|nil
	---#end

Maybe you would like to open a PR for it?