bugGNU Octave - Bugs: bug #46882, urlread: allowing https without...

 
 

bug #46882: urlread: allowing https without certificate validation

Submitted by:  Michael Hirsch <mhirsch>
Submitted on:  Tue 12 Jan 2016 08:31:39 PM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Matlab Compatibility
Status: ConfirmedAssigned to: None
Originator Name: Michael HirschOpen/Closed: Open
Release: devOperating System: Any

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Thu 14 Jan 2016 10:44:09 PM UTC, comment #6:

OK Mike, I can probably just do try-catch with a system() call to curl for now, although that won't work on Windows without Cygwin.
Thanks.

Michael Hirsch <mhirsch>
Thu 14 Jan 2016 05:53:22 PM UTC, comment #5:

Since --traditional (aka --braindead) only sets the state of warning messages and certain appearance preferences, changing behavior of this level would be out of scope for that option IMHO.

Mike Miller <mtmiller>
Project Administrator
Thu 14 Jan 2016 05:48:13 PM UTC, comment #4:

Thanks Mike, so it seems like that weboptions() no-validate option might also require a try-catch in user M-code since Matlab doesn't have such an option.

Would a satisfactory fix be to have urlread() disable host/cert verification when Octave is started with --braindead?

Michael Hirsch <mhirsch>
Thu 14 Jan 2016 05:32:12 PM UTC, comment #3:

I don't think curl supports overriding this with an environment variable, but adding an environment variable seems like the easiest way to allow users to do this if they want.

Or extend the weboptions() function to enable/disable host and/or certificate chain verification (after weboptions is implemented).

Mike Miller <mtmiller>
Project Administrator
Thu 14 Jan 2016 03:20:14 PM UTC, comment #2:

Thanks Oliver, yes I too think that allowing unvalidated certificate should be a user-selectable option, even though Matlab allows unvalidated https with urlread() by default.

Many servers 301 redirect from http to https. Two problems result:
1) redirect is incorrect (broken URL) and sysadmin has no time to fix edge cases
2) invalid certs

Matlab just works in case 2, the more common case.

Michael Hirsch <mhirsch>
Thu 14 Jan 2016 07:09:32 AM UTC, comment #1:

It is dangerous to ignore invalid certificates by default. Also it is likely to see more valid certificates in the future given that (1) more and more browsers refuse to show sites with invalid certificates and (2) it gets easier to install valid certificates (Let's Encrypt).

In general you would switch to unencrypted HTTP on any site with invalid certificates, because HTTPS would be pretty useless in these cases anyway. (Well, you could argue that you have at least some encryption which helps against non-intercepting(!) third-parties.)

However, it would be useful to have an option to ignore invalid certificates for sites that could otherwise not be accessed, e. g., if plain HTTP was not available and you don't want to set up an exception for that certificate.

Oliver Heimlich <oheim>
Tue 12 Jan 2016 08:31:39 PM UTC, original submission:

As more and more websites are rightly enabling HTTPS, some benign sites such as scientific data sites or intranet servers have self-certified certificates that Octave/libcurl chokes on with:
error: urlread: Peer certificate cannot be authenticated with given CA certificates

while Matlab urlread() and webread() read them fine. An example URL is:
https://amisr.asf.alaska.edu/
yielding the libcurl cert error.

I believe one could patch libinterp/corefcn/urlwrite.cc
with an option like:
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE)

as Matlab seems to not verify or care about authenticating certificates with webread() or urlread().

Michael Hirsch <mhirsch>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by oheim (Posted a comment)
  • -unavailable- added by mhirsch (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follows 1 latest change.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 14 Jan 2016 05:32:12 PM UTCmtmillerStatusNone=>Confirmed

    Back to the top


    Powered by Savane 3.1-cleanup1