Yes, this can be closed.  Abdallah did a great job this summer

https://abdallahshamy.wordpress.com/

Bugs / improvements of jsondecode, jsonencode can be handled in different bug reports.

Many thanks for introducing all your JSON projects here.  This discussion was very helpful 🙂

Closing as fixed.

I'm still working on JSONStuff - https://github.com/apjanke/octave-jsonstuff - and I think it's about feature-complete, but it's not Fast (because some stuff is still in the M-code layer, that should be down at the C++ oct-file layer). I'm working on adding more unit tests to it, and once I'm comfortable with the test coverage, will work on optimizing it with C++.

I'd like to get it on to Octave Forge at this point, if you're still taking Forge submissions. I think it's good enough for end users to work with. And while there are other Octave JSON packages out there, users might be interested in this one in particular since it is Matlab-compatible, so there's no need to modify M-code being ported from Matlab.

Hi all,

I think Mike suggestion is the way to go.
In other words, what is needed in json-stuff to separate the user interface with the low level library?

My first observation would be that the parsing of options should be done at the m-file level (so we do not need to re-do it if we change the low-level library)

For jsonencode we need to parse the 'ConvertInfAndNaN' option

I do not see any other option for jsondecode.

So what is stopping us from using json-stuff as the first working implementation of the functionality?

If the separation is well done, the decision of what is a good low level library (or even checking for different libraries, like https://github.com/nlohmann/json (MIT)) can be postponed.

Hi Qianqian! Thanks for chiming in.

I think that JSONlab and my jsonstuff package could happily co-exist. The goal of jsonstuff is simply to create Matlab-compatible implementations of jsonencode/jsondecode for Octave. My impression is that jsonencode/jsondecode is to allow Matlab to access external APIs that provide data in "native" JSON format. It doesn't cover more general or flexible JSON handling, or persisting of arbitrary Matlab data structures to JSON, like your JSONlab does. My goal is to eventually get jsonstuff merged into core Octave and cease to have it be an independent package.

I have no intention of expanding jsonstuff beyond the base jsonencode/jsondecode, or (probably) even supporting the Matlab Production Server JSON format, since that's not available in base Matlab, and as far as I know, MPS interoperability isn't an Octave design goal. jsonstuff's goals are: 1) Matlab-compatible jsonencode/jsondecode, 2) decent performance, and 3) nothing else.

I'll take a look at your JData stuff when I have some spare time; I'm pretty busy with the day job and a conference right now. Offhand, I think support for extended JSON functionality like that should probably stay independent from Octave's jsonencode/jsondecode stuff: since that's not provided by base Matlab either, it seems suited for a package that would work in both Matlab and Octave.

Just found this thread. Glad that someone is working on a builtin JSON encoder/decoder for Octave. My JSONLab toolbox (http://github.com/fangq/jsonlab), as mentioned by the OP, does work for Octave, but the speed to encode/decode large sized JSON data is horrific on Octave.


I just want to mention a JSON-base complex data encoding scheme - JData specification - that I have been working on. Most of these were derived from the encoding method I already used in my JSONLab toolbox, but generalized significantly. Happy to hear if you have any feedback/suggestions/thoughts

http://github.com/fangq/jdata

more background info can be found in this post

https://mailarchive.ietf.org/arch/msg/json/z7cUcdABZSS_sgvhVExWUW5y1Yc


I also want to point out jsonencode/jsondecode in MATLAB do not support saving/loading complex/sparse arrays, as well as some other complex data structures. I noticed that starting from MATLAB R2018a, matlab production server (mps) includes a more complex JSON encoder/decoder

https://www.mathworks.com/help/mps/restfuljson/json-representation-of-matlab-data-types.html

it also uses annotated struct mwtype/mwsize/mwdata to encode complex data, almost exactly like what I did in JSONLab and JData.

anyways, happy to hear if you have any thoughts on this project.

@apjanke - I wasn't suggesting that the code should support multiple backends at all, rather suggesting that you might look at the various options available before locking in on one. If possible, maybe design the code with some separation to make it feasible to port the functions to a different parser if the need arises someday.

You can continue to develop and stabilize the jsonstuff any way you want to, but ultimately I would hope to see them integrated into Octave's core library. It is up to you if you want to integrate into a package first and then into Octave, but eventually they should be pulled into Octave one way or another.

Linking with the libjsoncpp library is no problem, the license is clearly GPL compatible.

There seem to be no shortage of json parsing library options in C and C++ with different tradeoffs. If possible I would try to keep the options open to allow porting the functions to different parsing backends as development progresses. For example rapidjson sounds like a very appealing alternative.

Hi Philip!

> Are we sure the io package is a good home for it? If yes, good, if no, maybe miscellaneous could be better, I really don't know. Opinions?

To me, io seems like the best fit among the existing supported Octave Forge packages. io is for "Input/output in external formats", and JSON seems to qualify as an external format. io's already got support for Office formats, which seems somewhat analogous. miscellaneous seems to be mostly numeric stuff, and doesn't currently have any file format support that I can see.

> OTOH if the JSON library is not too big, comprises a limited nr. of files, and has a GPL compatible license, it can be included in, and shipped with, the io package. JuanPi and I did s/th similar for the clipper polygon clipping library in the geometry package.

This is the approach I'm taking with jsonstuff now, using the JsonCpp library. It's 400KB including both the tarball and decompressed source, and is only three source files when using the "amalgamated source" approach.

"Vendoring" the library like this is by far my preferred approach: installing the lib using Linux or Mac package managers is really easy. But getting it installed and available on Windows is a big pain. And I'd like to see all 3 big OSes supported.

JsonCpp's license is "Public Domain in jurisdictions where that is recognized, and MIT License elsewhere". I don't know if that counts as GPL-compatible. If it's not, switching to a different GPL-compatible library might be the best choice.

I think for now I'd like to continue developing jsonstuff as its own package where I have commit rights so I can iterate on it quickly. And then once it's feature-complete and stable maybe it could be moved to io or core Octave.

atcl: Here's a new package with jsonencode/jsondecode for you: https://github.com/apjanke/octave-jsonstuff.

I would be happy to follow Philip's suggestion and introduce these two functions in to the io package, and then migrate them to core after they have proved stable and useful.

Matlab has a habit of adding functions that persist for only a few releases before being renamed or even removed.  These two have the feel of functions that are going to eventually change.

I'd rather call it's a feature request.

The io package contains a (contributed) json2object function.
What if people who want to write jsonencode and jsondecode contribute it to the io package (I'll happily accept them) to mature there? Once stable enough they can be moved to core Octave, if the core devs agree of course.
The io package is more regularly updated than Octave releases.