bugGNU Octave - Bugs: bug #63199, [feature request] add a function...

 
 

bug #63199: [feature request] add a function to inverse eval() function

Submitter:  Yu Hongbo <yuhongbo>
Submitted:  Tue 11 Oct 2022 02:58:33 AM UTC
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Feature Request
Status:  Need Info Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * other Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment (Rich Markup)
   

Comment Type & Canned Response:
       

 

( Jump to the original submission )

Wed 12 Oct 2022 01:40:09 AM UTC, comment #14: 

Alphabetize the structs with orderfield if you're doing that.

Anonymous
Wed 12 Oct 2022 01:10:48 AM UTC, comment #13: 

Oh I got a typo. The save and load logic can handle function_handle as well, so I can accept this workaround.

>> a = {{struct}, {@struct}};
>> save ('tmp', '-text', 'a');
>> b =  struct2cell(load('tmp','-text')){1}

b =
{
  [1,1] =
  {
    [1,1] =

      scalar structure containing the fields:

  }

  [1,2] =
  {
    [1,1] = @struct
  }

}

Yu Hongbo <yuhongbo>
Wed 12 Oct 2022 01:06:40 AM UTC, comment #12: 

This save and load logic can handle all-data variants. But how about mixed-data-and-function-handle variants? In this scenario the save and load logic cannot get the consistent result, here is my code:

>> a = {{struct}, {@struct}};
>> # b is the same to {{struct}, {struct}}, so the result has been changed
>> b =  struct2cell(load('tmp','-text')){1}

b =
{
  [1,1] =
  {
    [1,1] =

      scalar structure containing the fields:

  }

  [1,2] =
  {
    [1,1] =

      scalar structure containing the fields:

  }

}

Do you have another workaround in this scenario?

Yu Hongbo <yuhongbo>
Tue 11 Oct 2022 05:04:49 PM UTC, comment #11: 

Hi there!

In Octave the option "-text" of the "save" and "load" [1] functions can be used to read/write the data as text.

a = {{struct}, {struct}};

save ('tmp', '-text', 'a');

b =  struct2cell(load('tmp','-text')){1};

If '-' is used as filename the data is written to a string instead of a file.

str = save ('-', 'a');

I believe that this method doesn't have the limitations of jsonencode/jsondecode.

[1] https://docs.octave.org/v7.2.0/Simple-File-I_002fO.html

Anonymous
Tue 11 Oct 2022 02:46:00 PM UTC, comment #10: 

I thought we were sharing our opinions and doing technical communication. If you thought I was arguing with you, I make an apologize to you now.

Yu Hongbo <yuhongbo>
Tue 11 Oct 2022 01:55:20 PM UTC, comment #9: 

I tried to be helpful with suggestion how to approach your original problem.
It took until comment #8 (and me politely asking 3 times) to identify something that might actually be a bug. But I won't waste my spare time arguing with you.
Maybe, someone else will be more successful communicating with you than myself.

Markus Mützel <mmuetzel>
Project Member
Tue 11 Oct 2022 12:53:09 PM UTC, comment #8: 

I understand that the result of "jsonencode() then jsondecode()" may change in cross-language situation. All users should accept it. For example, users won't give up using database with Octave although Octave is different to SQL.

Users transform data with data structure, so the unique-data-guarantee is actually important. My examples is about nested JSON objects and JSON arrays, and the jsondecode() function of Octave will condense some data structure, then users will use extra index to rebuild the expected data. That will be a mess.

And JSON text which are generated with the jsonencode() function of Octave still have this kind of problem, here is my example in Octave version:

>> jsondecode(jsonencode({struct, struct}))

ans =

  2x1 struct array containing the fields:

>> jsondecode(jsonencode({{struct}, struct}))

ans =
{
  [1,1] =

    scalar structure containing the fields:

  [2,1] =

    scalar structure containing the fields:

}

>> jsondecode(jsonencode({{struct}, {struct}}))

ans =

  2x1 struct array containing the fields:

And did you mean that "every problem about every JSON text which are not generated with the jsonencode() function of Octave won't be considered as bug"?

Yu Hongbo <yuhongbo>
Tue 11 Oct 2022 10:49:50 AM UTC, comment #7: 

I might have misunderstood your original issue.
I understood you'd like to transfer variable values from a remote instance running Octave to a local instance.
In that scenario, you could use `jsonencode` on the remote machine to prepare the data for transmission and `jsondecode` on the local machine to recover the original data.
Every JSON string that you'd receive would be created by `jsonencode`. `jsondecode` would only be used on JSON strings created by `jsonencode`. So, only roundtrips from Octave value -> JSON string -> Octave value would need to recover the original value. You wouldn't receive "arbitrary" JSON encoded data.

I don't know if there is any guarantee that differing JSON strings would be decoded to unique data. (And I don't think you'd need that guarantee.)

Markus Mützel <mmuetzel>
Project Member
Tue 11 Oct 2022 09:20:55 AM UTC, comment #6: 

I wrote three diffenrent JSON data and did not use jsonencode() function. Why care about which jsonencode command? This is a legal JSON text.

The document is at https://docs.octave.org/v7.2.0/JSON-data-encoding_002fdecoding.html

Yu Hongbo <yuhongbo>
Tue 11 Oct 2022 08:58:09 AM UTC, comment #5: 

If users need to handle function handles specifically with your suggested approach anyway, they could also use, e.g., `func2str` on the remote side before creating the JSON string.

> the results should be used to rebuild the original data.

Please let me rephrase my question: What was the original data that created these JSON strings with `jsonencode`?

Sorry, I don't understand: What do you mean by "the document"?

Markus Mützel <mmuetzel>
Project Member
Tue 11 Oct 2022 08:48:17 AM UTC, comment #4: 

bug #1: eval() function will parse an expression, even this expression is meant to be a function handle. Users will consider the context when they need to use eval() function, so developers don't need to help making this kind of desicion. I give a scenario about transferring any kind of Octave variants:
Remote procedure call (RPC) is a distributed computing technology, it is used to transfer a procedure to another device. The current jsonencode() and jsondecode() logic cannot transfer any kind of Octave variants, so Octave cannot fully transfer the current to another remote Octave instance with the two functions.

bug #2: I expect

>> jsondecode('[{},{}]')

and

>> jsondecode('[[{}],{}]')

and

>> jsondecode('[[{}],[{}]]')

should return different results, and the results should be used to rebuild the original data.
Why care about which jsonencode command? This is a legal JSON text.

bug #3: The document only mentioned null, inside a numeric array and null, inside a non-numeric array. So how about null outside an array? I don't see this document.

Yu Hongbo <yuhongbo>
Tue 11 Oct 2022 07:05:07 AM UTC, comment #3: 

Wrt to your bug #1: Transferring function handles as text strings will always be problematic. They depend on the scope where they were created. E.g., they might refer to a local or private function, or be an anonymous function that stores variable values in its context. I don't see how your proposed function would help with that.

Wrt to your bug #2: Could you please clarify why that is unexpected? Which jsonencode command did you use to create these JSON strings?

Wrt to your bug #3: That is documented behavior. What you might be preferring:

>> jsonencode(NA, "ConvertInfAndNaN", false)
ans = NaN
>> jsondecode(ans)
ans = NaN

Markus Mützel <mmuetzel>
Project Member
Tue 11 Oct 2022 06:39:35 AM UTC, comment #2: 

Your idea is great(use the existing functions to avoid adding a new function), but there're some problems about the two functions.

The jsonencode() and jsondecode() function have undocumented transformation, unsupported transformation, unexpected transformation and array reshaping situations. So in this way Octave cannot transform every kind of variants to string.

Since the jsonencode() and jsondecode() function have Matlab versions, Octave should better add a new expression() function to leave the two functions compatible with Matlab.

Some bugs(if Matlab does so, we can do nothing but realize a new function) are as below:

# bug 1: unsupported transformation on jsonencode() function

>> jsonencode(@source)

error: jsonencode: unsupported type

# bug 2: unexpected transformation on jsondecode() function

>> jsondecode('[{},{}]')

ans =

  2x1 struct array containing the fields:

>> jsondecode('[[{}],{}]')

ans =
{
  [1,1] =

    scalar structure containing the fields:

  [2,1] =

    scalar structure containing the fields:

}

>> jsondecode('[[{}],[{}]]')

ans =

  2x1 struct array containing the fields:

# bug 3: undocumented transformation on jsondecode() function

>> jsondecode(jsonencode(NA))

ans = [](0x0)
# the document doesn't contain "null, outside an array"

Yu Hongbo <yuhongbo>
Tue 11 Oct 2022 04:15:14 AM UTC, comment #1: 

Please take a look at jsonencode and jsonencode:
https://docs.octave.org/v7.2.0/JSON-data-encoding_002fdecoding.html

Markus Mützel <mmuetzel>
Project Member
Tue 11 Oct 2022 02:58:33 AM UTC, original submission:  

With the popularization of distributed software development, we are more inclined to give complex operations to the server,  retrieve the results through Web services, then parse the results   to variants.

However, the retrieved results mostly cannot directly use the printed text. For example,

a =
{
  [1,1] = 1
  [1,2] =
  {
    [1,1] = 2
  }

}

Octave cannot elegently parse this printed text to the original cell. A good way is add a function to generate expressions which can be handled with exec() function.

So I propose to add expression() function, and the usage is like:

>> ANS = exec(expression(VARIANTS));
>> # the ANS here is a copy of VARIANTS


For distributed system, the example is like:

>> request_body = webread(website, options);
>> ANS = exec(request_body);
>> # the request_body here is the expression which the web server intents to response, which compliants to expression(VARIANTS)
>> # the ANS here are parsed variants


Yu Hongbo <yuhongbo>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by yuhongbo (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only project members can vote.

     

     

     

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-10-11 mmuetzel Carbon-CopyRemoved 102357 -
    2022-10-11 mmuetzel Carbon-CopyRemoved 102357 -
    2022-10-11 mmuetzel StatusNone Need Info

    Back to the top


    Powered by Savane 3.9