bugPSPP - Bugs: bug #18692, Better integration of URLs in...


bug #18692: Better integration of URLs in system/portable file specifications

Submitter:  Ben Pfaff <blp>
Submitted:  Thu 04 Jan 2007 11:57:37 PM UTC
Category:  System/Portable File I/O Severity:  1 - Wishlist
Status:  None Assigned to:  None
Open/Closed:  Open Release:  Future
Effort:  0.00
* Mandatory Fields

Add a New Comment Rich Markup

Mon 30 Jul 2007 05:10:11 PM UTC, comment #3: 

I checked in that patch.  Marking this post-0.6.0 now, updating title.

Ben Pfaff <blp>
Group administrator
Mon 30 Jul 2007 07:12:32 AM UTC, comment #2: 

I'd be happy with that for now.  We can consider the other alternatives later.

John Darrington <jmd>
Group administrator
Mon 30 Jul 2007 03:49:43 AM UTC, comment #1: 

I am inclined to fix this bug by adding a suggestion to the documentation that one can read data directly from a URL by using a pipe file name that uses a helper program, e.g. "|curl http://janda.org/c10/data%20sets/vote92/vote92.por".  A patch that implements this, and fixes a few bugs I encountered while testing it, is attached.

I considered two alternatives:

  • Link against curl or libwww (from w3c), as jmd suggested.  Long term, this might be a good solution.  The problem is that we'd have to add an additional layer of wrappers over stdio to make this reasonable for the client code.  I can certainly do that, but I'm slightly concerned about making it efficient in the common case (though I don't have any evidence that it would be inefficient), readable, and simple all at the same time.  Also, there is an impedance mismatch between curl/libwww and PSPP: curl wants to provide data through callbacks, which isn't convenient, so we'd have to do buffering. 

  • Fork a "curl" or "wget" process and communicate with it over a pipe.  The only problem here is that mingw32 doesn't have fork, and I'd like to make this a general-purpose solution.  We could use popen, which is implemented by mingw32, but then error reporting would suffer: the subprocess would only be able to report errors via its exit status (popen doesn't provide a way to intercept stderr).

(file #13525)

Ben Pfaff <blp>
Group administrator
Thu 04 Jan 2007 11:57:37 PM UTC, original submission:  

Suggestion from Usenet:

> From: "Mike Lacy" <Michael.Lacy@colostate.edu>
> Newsgroups: comp.soft-sys.stat.spss
> Subject: How to read data file directly from a URL
> Date: 4 Jan 2007 14:19:05 -0800
> Organization: http://groups.google.com
> Message-ID: <1167949144.995682.126880@51g2000cwl.googlegroups.com>
> Greetings old friends--- haven't posted in a while because I'm using
> SPSS less and because I've had to start using the Google groups
> interface, which annoys me.
> On topic:
> Is there any convenient way to use SPSS read a data file directly from
> a URL?  I'm thinking of something like:
> get file = "http://www.myspssdatafiles/GSS2004.sav" .
> I suppose something like this could be implemented via OBDC, but that
> doesn't seem very easy.  At least Stata (R?) has the equivalent syntax,
> and I was hoping the same thing could be done some way in SPSS.
> Regards,
> Mike Lacy

From: Jason Stover <jhs@math.gcsu.edu>

>R can do this. It makes lab demonstrations
>very easy for students.

From: John Darrington <john@cellform.com.au>

>It would be very easy to implement, if we link with libcurl.  >Let's
>add it as a feature which get's automatically compiled in if >autoconf
>detects that libcurl is available.

Ben Pfaff <blp>
Group administrator


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

Attached Files
file #13525:  url-files2.patch added by blp (5KiB - text/x-diff)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by jmd (Posted a comment)
  • -email is unavailable- added by blp (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.


    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-05-11 jmd ReleasePost-0.6.0 Future
    2007-07-30 blp ReleaseBefore 0.6.0 Post-0.6.0
    2007-07-30 blp StatusReady for Test/Review None
        SummaryAllow URLs in system/portable file specifications Better integration of URLs in system/portable file specifications
    2007-07-30 blp Attached File- Added url-files2.patch, #13525
        StatusNone Ready for Test/Review
    2007-07-25 blp ReleaseNone Before 0.6.0

    Back to the top

    Powered by Savane 3.13-4b48.
    Corresponding source code