bugGNU Octave - Bugs: bug #46882, urlread: allowing https without...

 
 

bug #46882: urlread: allowing https without certificate validation

Submitter:  -*- <mhirsch>
Submitted:  Tue 12 Jan 2016 08:31:39 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  Confirmed Assigned to:  None
Originator Name:  Michael Hirsch Open/Closed:  * Open
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 14 Jan 2016 10:44:09 PM UTC, comment #6: 

OK Mike, I can probably just do try-catch with a system() call to curl for now, although that won't work on Windows without Cygwin.
Thanks.

-*- <mhirsch>
Thu 14 Jan 2016 05:53:22 PM UTC, comment #5: 

Since --traditional (aka --braindead) only sets the state of warning messages and certain appearance preferences, changing behavior of this level would be out of scope for that option IMHO.

Mike Miller <mtmiller>
Group Member
Thu 14 Jan 2016 05:48:13 PM UTC, comment #4: 

Thanks Mike, so it seems like that weboptions() no-validate option might also require a try-catch in user M-code since Matlab doesn't have such an option.

Would a satisfactory fix be to have urlread() disable host/cert verification when Octave is started with --braindead?

-*- <mhirsch>
Thu 14 Jan 2016 05:32:12 PM UTC, comment #3: 

I don't think curl supports overriding this with an environment variable, but adding an environment variable seems like the easiest way to allow users to do this if they want.

Or extend the weboptions() function to enable/disable host and/or certificate chain verification (after weboptions is implemented).

Mike Miller <mtmiller>
Group Member
Thu 14 Jan 2016 03:20:14 PM UTC, comment #2: 

Thanks Oliver, yes I too think that allowing unvalidated certificate should be a user-selectable option, even though Matlab allows unvalidated https with urlread() by default.

Many servers 301 redirect from http to https. Two problems result:
1) redirect is incorrect (broken URL) and sysadmin has no time to fix edge cases
2) invalid certs

Matlab just works in case 2, the more common case.

-*- <mhirsch>
Thu 14 Jan 2016 07:09:32 AM UTC, comment #1: 

It is dangerous to ignore invalid certificates by default. Also it is likely to see more valid certificates in the future given that (1) more and more browsers refuse to show sites with invalid certificates and (2) it gets easier to install valid certificates (Let's Encrypt).

In general you would switch to unencrypted HTTP on any site with invalid certificates, because HTTPS would be pretty useless in these cases anyway. (Well, you could argue that you have at least some encryption which helps against non-intercepting(!) third-parties.)

However, it would be useful to have an option to ignore invalid certificates for sites that could otherwise not be accessed, e. g., if plain HTTP was not available and you don't want to set up an exception for that certificate.

Oliver Heimlich <oheim>
Tue 12 Jan 2016 08:31:39 PM UTC, original submission:  

As more and more websites are rightly enabling HTTPS, some benign sites such as scientific data sites or intranet servers have self-certified certificates that Octave/libcurl chokes on with:
error: urlread: Peer certificate cannot be authenticated with given CA certificates

while Matlab urlread() and webread() read them fine. An example URL is:
https://amisr.asf.alaska.edu/
yielding the libcurl cert error.

I believe one could patch libinterp/corefcn/urlwrite.cc
with an option like:
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE)

as Matlab seems to not verify or care about authenticating certificates with webread() or urlread().

-*- <mhirsch>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by runejuhl
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by oheim (Posted a comment)
  • -email is unavailable- added by mhirsch (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-09-17 runejuhl Carbon-Copy- Added runejuhl
    2016-01-14 mtmiller StatusNone Confirmed

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code