bugGNU Parallel - Bugs: bug #49843, --csv as output for GNU sql

 
 

bug #49843: --csv as output for GNU sql

Submitter:  Ole Tange <tange>
Submitted:  Wed 14 Dec 2016 05:59:52 PM UTC
   
 
Category:  None Severity:  1 - Wish
Priority:  5 - Normal Item Group:  None
Status:  Confirmed Privacy:  Public
Assigned to:  None Open/Closed:  Open
* Mandatory Fields

Post a Comment

Add a New Comment Rich Markup
   

Discussion

Fri 16 Dec 2016 02:08:06 AM UTC, comment #1: 

A better approach is to use an SQLite-base and then convert that to CSV:

         DBURL=sqlite3:///%2Ftmp%2Fmy.sqlite
         DBTABLEURL=$DBURL/mytable
         parallel --sqlandworker $DBTABLEURL seq ::: {1..10}
         sql $DBURL '.headers on' '.mode csv' 'select * from mytable;'

       This takes around a second per job.

       If you have access to a real database system, such as PostgreSQL, it is even
       faster:

         DBURL=pg://user:pass@host/mydb
         DBTABLEURL=$DBURL/mytable
         parallel --sqlandworker $DBTABLEURL seq ::: {1..10}
         sql $DBURL "COPY (SELECT * FROM mytable) TO stdout DELIMITER ',' CSV HEADER;"

       Or MySQL:

         DBURL=mysql://user:pass@host/mydb
         DBTABLEURL=$DBURL/mytable
         parallel --sqlandworker $DBTABLEURL seq ::: {1..10}
         sql -p -B $DBURL "SELECT * FROM mytable;" > mytable.tsv
         perl -pe 's/"/""/g;s/\t/","/g;s/^/"/;s/$/"/;s/\\\\/\\/g;s/\\t/\t/g;s/\\n/\n/g;' mytable.tsv

Ole Tange <tange>
Group administrator
Wed 14 Dec 2016 05:59:52 PM UTC, original submission:  

sql --csv $DBURL/table

Ole Tange <tange>
Group administrator

 

Attached Files

This item currently has no attached files.

(Note: upload size limit is set to 4.0MiB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

 

Dependencies

This item does not depend on any other items.

No items depend on this one.

 

Mail Notification Carbon-Copy List

Carbon-Copy List
  • -email is unavailable- added by tange (Submitted the item)
  •  

    Votes

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    History

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.16-1b88.
    Corresponding source code