I used grep to find a list of isfinite instances where the output is negated (and hence could be large if the input were sparse).  The instances are

+matlab/+lang/MemoizedFunction.m:134:                  || ! isfinite (val))
+matlab/+lang/MemoizedFunction.m:140:              if (! isscalar (val) || ! isnumeric (val) || ! isfinite (val) ...
general/int2str.m:80:  nan_inf = ! isfinite (n(:));
general/quadgk.m:374:      if (any (! isfinite (q_subs)))
general/quadgk.m:450:      if (any (! isfinite (q_subs)(:)))
general/quadl.m:112:  elseif (any (! isfinite (q(:))))
general/quadv.m:154:  elseif (any (! isfinite (q(:))))
linear-algebra/cond.m:67:    elseif (any (! isfinite (A(:))))
miscellaneous/mustBeInteger.m:53:  elseif (any (! isfinite (x)))
optimization/glpk.m:526:  if (any (! isfinite (b(:))))
optimization/glpk.m:593:            || any (! isfinite (sense)))
polynomial/roots.m:89:  elseif (any (! isfinite (c)))
set/uniquetol.m:207:      eq_rows(i,1) = eq_rows(i,1) || any (! isfinite (Arow_i), 2);
specfun/primes.m:55:  if (! isfinite (n) && n != -Inf)
testfun/assert.m:351:                      | ! isfinite (A_null_real) & ! isfinite (B_null_real);
testfun/assert.m:362:                        | ! isfinite (A_null_imag) & ! isfinite (B_null_imag);

Most of these can be ignored because the input can be guaranteed to be small.  I changed cond.m and mustBeInteger.m in this changeset: http://hg.savannah.gnu.org/hgweb/octave/rev/303bbdc3c536.

The remaining question is whether to change the two instances in assert.m  It will require introducing two new temporary variables, making their calculation dependent on whether the input is full or sparse, and then clearing those variables at the end of the block doing this calculation.

the fix was related to this problem identified through the help mailing list:

https://lists.gnu.org/archive/html/help-octave/2022-10/msg00011.html

I'm guessing your input to `glpk` is sparse. Line 507 of `glpk.m` uses:
  if (any (! isfinite (A(:))))
    error ("glpk: The values in A must be finite");
  endif

`isfinite (A(:))` creates a sparse Boolean matrix. However, it is likely not
sparsely populated which might be the cause for the memory issue.

Does it work if you edit that file and change the above lines to the following?
  if (any (isinf (A(:))) || any (isnan(A(:))))
    error ("glpk: The values in A must be finite");
  endif

that turned out to solve the problem for that user, and so was pushed as a potential fix to prevent reoccurrence.

In changeset 800eb86438cc a modification was made to avoid using isfinite() on sparse input.  I'm not sure what problem that was addressing, but it did lead me to find a difference between Octave and Matlab.

Sample code:

MATLAB

x = sparse ([Inf, 1; 2 NaN]);
isfinite (x)
ans = 2x2 logical array
1 0
0 1

Octave

ans =

Compressed Column Sparse (rows = 2, cols = 2, nnz = 2 [50%])

  (2, 1) -> 1
  (1, 2) -> 1

I don't know if we want to follow Matlab or not.  It seems like this is inconsistent with isinf() and isnan() functions which do return a sparse output for sparse input in Matlab.  Octave is arguably being correct here by keeping that pattern.