Add a New Comment (Rich Markup)
Adding a link from bug #58953 that might be useful to deal with m.Data(end): https://stackoverflow.com/questions/23016606/what-are-the-semantics-of-end-in-matlab/23017087#comment35192810_23017087 The issue here is that m.Data(end) will first evaluate m.Data, i.e. read the entire mapped data into memory and then return its last value, defeating the point of memory mapping. Or is it that the dependent property Data should contain an object from the _memmapfile_handle_ handle class (instead of private property MemoryMap) so that an end() method can be implemented in that class?
As a follow-up, I attach a version of the toy example I mentioned previously, _mmap_.cc, rewritten as a DEFUN_DLD. It works like this:
octave> fid = fopen ("data.dat", "w"); octave> fwrite (fid, 1:128, "uint8"); octave> fclose (fid); octave> octave> h = __mmap__ ("data.dat", 128, 0); % mmap octave> __mmap__ (h, "uint8", 0:7) ans = 1 2 3 4 5 6 7 8 octave> typecast (ans, "double") ans = 5.4476e-270 octave> __mmap__ (h, "double", 0) ans = 5.4476e-270 octave> __mmap__ (h, 128); % munmap
One question I have pertains to subsref/subsasgn: instead of having my own inefficient implementation (speed- and memory-wise), is there a way to reuse Octave's versions? I.e., given:
void *ptr; /* from mmap */ dim_vector dim; /* array dimensions */ std::string precision; /* eg, "uint32" */ octave_value_list idx; /* eg, {":", [3 4 5], 2, ":", [2 3]} */
would return a uint32NDArray containing the values from *prt corresponding to idx.
Matlab has a class to perform memory-mapping on a file from disk: https://www.mathworks.com/help/matlab/ref/memmapfile.html I attach two files, memmapfile.m and _memmapfile_handle_.m, as an initial attempt at implementing this functionality in Octave and would welcome feedbacks and help to finish it. As far as I can see, the code in memmapfile.m is feature complete. The internal handle class in _memmapfile_handle_.m is where further work is required. An extra handle class has to be defined because memmapfile is a value class and, given that the memory mapping can take place at any time including when the data is first accessed for reading, a mechanism has to be in place to store the mapping pointer. At the moment, memory mapping isn't actually taking place and fread/fwrite is used to mock the real behavior (I know, disappointing!). For example, instead of having: this.handle = fopen (filename, writable); there should be a call: [this.handle, err, msg] = _mmap_ (filename, length, offset, writable); to a low-level (oct-file or core?) _mmap_ that would open/mmap/close the file and return the address of the mapping as a double. This function would also have to deal with page size for the offset parameter so it would probably need to have extra output variables containing the eventually modified offset and length of the mapping. Same thing with munmap instead of fclose. I have a toy example showing how memory mapping would work but I need help/guidance in specifying and writing the DEFUN_DLD function(s). Other aspects to consider:
https://docs.microsoft.com/en-us/windows/desktop/memory/file-mapping
Thanks to jwe and carandraug for their help.
(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)
Attach Files: Comment:
Depends on the following items: None found
Items that depend on this one: None found
There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.
Only project members can vote.
Please enter the title of George Orwell's famous dystopian book (it's a date):
Follow 3 latest changes.
Copyright © 2023 Free Software Foundation, Inc. Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved. The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team. Source Code
Powered by Savane 3.11