taskGNU Astronomy Utilities - Tasks: task #13786, Usable libraries for other...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #13786: Usable libraries for other languages

Submitter:  Mohammad Akhlaghi <makhlaghi>
Submitted:  Sat 24 Oct 2015 01:02:20 PM UTC
   
 
Should Start On:  Fri 23 Oct 2015 03:00:00 PM UTC Should be Finished on:  Fri 23 Oct 2015 03:00:00 PM UTC
Category:  Libraries Priority:  7 - High
Item Group:  Enhancement Status:  Postponed
Privacy:  Public Assigned to:  None
Percent Complete:  10% Open/Closed:  Open
Effort:  0.00

Jump to the original submission

Sat 28 Mar 2020 08:38:23 PM UTC, comment #19: 

No, I am not going into that detail now, but this question just got my attention? that's why asked.
This link contains about what I asked.
link to https://docs.python.org/3/extending/newtypes_tutorial.html
In it see the "note" written.

But now I got it, I should try to link it with numpy.
Thanks

madhav_bansal <madhav_bansal>
Sat 28 Mar 2020 08:29:24 PM UTC, comment #18: 

At the initial GSoC proposal level, do you have to go to such low-level implementation details?

It would be better if you include a link to the respective Python documentation, so we can see exactly what you are referring to ;-).

The basic idea is that Numpy (like Python) is written in C. Numpy's main data structure (which is a C structure) is very similar to our `gal_data_t' (but much more general). So, the way I see it now is that the whole conversion function should be in C: you don't want to invent an alternative to Numpy ;-). Once the two way "Numpy <---> gal_data_t" converter is written (fully in C), we can start writing C functions that interface with Python.

But this is just my general view now, I have only read about it, probably during implementation some things will be different.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sat 28 Mar 2020 07:49:17 PM UTC, comment #17: 

hey, I have a doubt , we have to write an extension type for GNUASTRO builtin struct types.(like -Gal_data_t).Python official documentation says that we can write a static extension type.What I want to ask is we are using GNUASTRO libraries as shared libraries, so can we use static extension types?
Actually,I am bit confused about what does the "statics extension types" and "heap-allocated extension types(dynamically allocated)" mean?

madhav_bansal <madhav_bansal>
Sat 28 Mar 2020 10:48:13 AM UTC, comment #16: 

Is that task enough or should I find more ?

Kartik Ohri <lucifer13>
Wed 25 Mar 2020 05:18:01 PM UTC, comment #15: 

Thanks Kartik, that is great! I think if you focus on Julia it will have a higher chance of acceptance: Julia and Java are very different things and not much related. You can say that a simple conversion will take two months, but that you will work on improving Gnuastro's internal library also during the work, in a way that is more friendly to wrappers.

For example, one big improvement can be to define the existing programs as library functions, so users of other languages don't need to use system calls (which are heavy and a separate process and need inputs as files, not as in-memory arrays). This can also be useful for C/C++ users, not just high-level languages.

Ofcourse, this also applies to the Python libraries, or any other high-level language ;-).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Wed 25 Mar 2020 10:49:21 AM UTC, comment #14: 

I am planning to write a Julia wrapper for GNUAstro. Mose has already ensured that GNUAstro compiles on all platforms Julia supports. I intend to provide a direct interface to access the C library for those who are interested and a Julia style interface as well for others.

I think a two month time span is reasonable enough to accomplish this. In the spare time, I think I can either do a Java wrapper or complete some tasks to improve GNUAstro iteslf.

makhlagi, is a java wrapper desirable ? Do you think this is a good plan ?

Kartik Ohri <lucifer13>
Tue 24 Mar 2020 11:11:04 PM UTC, comment #13: 

Adding Kartik and Mosè for added discussions on Julia wrappers.

Just note that this is a general discussion on wrappers (ideally in any language). If the discussion is too specific to one language, we can define it as a new task ;-).

So far we haven't actually started so there isn't any tasks. If you have already started (or plan to start soon), please add the respective task and continue the language-specific details there.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Fri 20 Mar 2020 02:53:27 PM UTC, comment #12: 

Yes, even the users of Gnuastro's C library don't have access to the functions declared in the headers of `lib/gnuastro-internal'.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Fri 20 Mar 2020 11:38:47 AM UTC, comment #11: 

What I wanted to know is that if write wrapper functions for the files in lib/gnuastro, we don't have to write wrapper for these files?

madhav_bansal <madhav_bansal>
Fri 20 Mar 2020 11:13:26 AM UTC, comment #10: 

Both contain headers. The headers in `lib/gnuastro' get installed (as art of the library and are well documented in the book!). The ones in `lib/gnyastro-internal' don't get installed: they are only for "internal" Gnuastro operations while its being built. Their documentation is only as comments in the code.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Fri 20 Mar 2020 10:43:23 AM UTC, comment #9: 

In library folder of  gnuastro we have many files and two more sub-folder,one is name gnuastro which contains the header files, but the other one named gnuastro-internal what is that?

madhav_bansal <madhav_bansal>
Thu 19 Mar 2020 02:59:23 PM UTC, comment #8: 

Yes ;-)! So far, I think this it the cleanest and lowest-level way: at the start of the function we have some special functions to convert framework to C (while keeping the originally allocated array: not making an extra copy), do the main job in C through any number of calls to any library, and then convert the output data structures back to Python and return them.

Python is a language that evolves/changes a lot! Numpy evolves even faster than Python!!! So if you want your work to used in a more generic/longer-term manner, it is best to minimize the exposure of your work to this changing environment and just update the conversion tools when they change.

However, as you go deeper into it (during the implementation), you may find that for some cases/situations, it may be better to make Python classes, you'll definitely figure this out as you go ahead during implementation ;-).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Thu 19 Mar 2020 02:26:55 PM UTC, comment #7: 

I am just confirming if i understood it correct.
So what i have understood is that we do not have to make python classes to mirror our C structs right? but we have to convert
these C structs to NumPy's data container?

madhav_bansal <madhav_bansal>
Thu 19 Mar 2020 01:59:56 PM UTC, comment #6: 

As you have cited, the core of the work is writing a function to convert `gal_data_t' to Numpy's data container structure.

Then functions that are called by Python can follow this scenario: call this function to convert the data container (without changing the array pointers ideally: to allocate more space), call the Gnuastro library functions in any specific order, then convert them back to Numpy's data container and return it to Numpy/Python.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Thu 19 Mar 2020 01:50:18 PM UTC, comment #5: 

yes, I have already read that link , but this shows basics for wrapping the c function and C constants and raising custom errors. but it doesn't help with using our defined  C
 structs. Do we have to create mirror C structures?

madhav_bansal <madhav_bansal>
Thu 19 Mar 2020 01:42:37 PM UTC, comment #4: 

I hope you find this link useful. It is a basic quick tutorial for this project and might be useful for your specific query as well;-)

Sachin Kumar Singh <sks_15>
Thu 19 Mar 2020 01:22:21 PM UTC, comment #3: 

we can write python wrappers for the c function in which first the parse arguments to convert them from python object to related c types to be passed to our c function. but there are many function in gnuastro lib that takes gal_data_t and other our defined c structs and also returns pointers to gal_data_t type structs....how can we convert them to related to python objects

madhav_bansal <madhav_bansal>
Sun 04 Sep 2016 10:32:36 AM UTC, comment #2: 

Following the Libtool guidelines, the library header files were corrected to also be usable in C++. The change has been pushed to my personal (development) repository. As you see in the Libtool page, this is much more comprehensive than the suggested addition in the first comment of this task.

A small test C++ program was also added (in `tests/lib/versioncpp.cpp'). It will just do the simplest possible job of printing the version number of Gnuastro, but the important thing is that it is done within C++ which would fail with the headers before this commit.

I tried temporarily renaming the `g++' executable file to see what would happen if we don't have a C++ compiler, but apparently Autoconf is too smart for that and was able to use `gcc' for the C++ compilation. I currently don't have much experience with C++, but if it is possible to only have a C compiler (without C++), then it would be great if we could detect that at configure time and skip this (and possible future) test(s) if the user can't compile C++.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Thu 26 May 2016 06:15:04 AM UTC, comment #1: 

While SWIG is a generic tool, for better preformance and customizability. For the more commonly used languages, it might be better to write the converting files ourselves using Python and NumPy's own low-level C APIs:

https://docs.python.org/3/c-api/index.html

http://docs.scipy.org/doc/numpy-1.10.0/reference/c-api.html

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sat 24 Oct 2015 01:02:20 PM UTC, original submission:  

After task #13765 (Convert static libraries to shared libraries) is done, add the proper tools to allow the libraries to be used by other languages.

Most pipelines today use Python and most programmers use C++, being a very low-level tool, Gnuastro should be available to them also, not just C users. So it would be great to allow the users of these (and other languages) also use Gnuastro's libraries.

For C++ it is very streightforward, just add the lines below on the top and bottom of the header files:

Top:
#ifdef __cplusplus
extern "C"{
#endif


Bottom:
#ifdef __cplusplus
}
#endif



For other languages, SWIG seems to be an interesting solution. From what I understood of SWIG, we can even write a specific Makefile for making shared libraries for any language that it supports. So for example as options to "./configure". The user can run something like "./configure --forpython --forperl". Then the Makefile can run SWIG in the proper way and generate the shared library that can be used in that language. Finally "make install" can install those libraries besides the actual libraries and executable programs so the user can use Gnuastro in what-ever languate they like.

If we have enough interested people to help, we can even write test-suites for these libraries.

Mohammad Akhlaghi <makhlaghi>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by makhlaghi
  • -email is unavailable- added by sks_15 (Posted a comment)
  • -email is unavailable- added by madhav_bansal (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-03-24 makhlaghi Carbon-Copy- Added lucifer13
        Carbon-Copy- Added giordano
    2020-03-19 makhlaghi SummaryUsable libraries for other languages (maybe using SWIG) Usable libraries for other languages
    2016-09-04 makhlaghi Percent Complete0% 10%
    2015-10-24 makhlaghi Item GroupNone Enhancement
        StatusNone Postponed

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code