Sat 20 Feb 2016 03:38:02 AM UTC, comment #3:
Repmat already returns sparse output for sparse input.
A big weakness with the bsxfun patch is that it builds the sparse matrix column at a time. This will probably involve copying the entire matrix for each row processed. That can be avoided if we know the number of non-zero elements in the final matrix.
Can someone suggest a good heuristic for estimating the number of non-zero elements in the solution? Ideally, I don't want too many special cases for different values of
(operator, operand1-shape/sparsity, operand2-shape/sparsity).
|
Sat 20 Feb 2016 03:21:37 AM UTC, comment #2:
The attached patch should allow bsxfun to work. It is a two-line patch relative to the patch supplied for bug #45219 and bug #40089, but that patch hasn't yet been applied.
We don't always want to return a sparse matrix from bsxfun for a mix of sparse and full. For example, @plus and @minus should generally return a full matrix. The attached patch skips one level of optimization for sparse matrices (maybe_optimized_builtin). The alternative would be to enhance the first stage of optimization to consider which operations should convert full to sparse and which should convert sparse to full.
Dan, could you put this patch through its paces, like timing it and checking it with 64 bit indexing? I only tried it on a cut-down example that fits in 32 bits.
I'll look into repmat next.
Note: For anyone trying to reproduce this, replace "rep" by "broad" in the definition of toobig.
(file #36404)
|
Wed 17 Feb 2016 09:50:39 AM UTC, comment #1:
The first problem is bsxfun giving a full matrix as output
Then, of course repmat(broad,...) gives a full matrix, what else?
|
Wed 17 Feb 2016 01:17:31 AM UTC, original submission:
32-bit Octave has a limit on the maximum number of elements in an array. I have recompiled from source (following the script at https://github.com/calaba/octave-3.8.2-enable-64-ubuntu-14.04 ), and now have 64-bit indexing.
Nevertheless, when I attempt to perform elementwise multiplication using a broadcast function, I get error: out of memory or dimension too large for Octave's index type
My suspicion is that this is a bug with repmat, and that instead of producing a sparse matrix it generates a full one. But I don't know Octave's internals well enough to be sure.
Minimal code to reproduce the problem:
My current workaround is to do the multiplication inside of a loop, but it's sloooooow:
Copying from my SO question:
http://stackoverflow.com/questions/35421967/dimension-too-large-error-when-broadcasting-to-sparse-matrix-in-octave
|