helpGNU Common Lisp - Support: sr #104164, Is it possible to call Windows SDK...

 
 

sr #104164: Is it possible to call Windows SDK from GCL?

Submitted by:  Joe Lee <joe9404>
Submitted on:  Tue 03 May 2005 02:28:13 PM UTC  
 
Category: NonePriority: 5 - Normal
Severity: 3 - NormalStatus: Done
Privacy: PublicAssigned to: None
Open/Closed: ClosedOperating System: Microsoft Windows

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Thu 21 Nov 2013 09:34:14 PM UTC, comment #7:

Just a final note here of an improved interface in git master, defdlfun

Camm Maguire <camm>
Project Administrator
Thu 05 May 2005 07:00:27 PM UTC, comment #6:

To: Joe Lee <savannah-bounces@gnu.org>
Cc: Joe Lee <cf9404@yahoo.com.tw>,
Mike Thomas <miketh@brisbane.paradigmgeo.com>,
-unavailable-
Subject: Re: [sr #104164] Is it possible to call Windows SDK from GCL?
References: <20050503-142814.sv40977.33421@savannah.gnu.org>
<20050504-025642.sv6190.38427@savannah.gnu.org>
<20050504-134211.sv3824.77408@savannah.gnu.org>
<20050504-134934.sv3824.6802@savannah.gnu.org>
<20050504-222339.sv6190.75725@savannah.gnu.org>
<20050505-123612.sv40977.79984@savannah.gnu.org>
--text follows this line--
Greetings!

Joe Lee <savannah-bounces@gnu.org> writes:

> Follow-up Comment #5, sr #104164 (project gcl):
>
> Sorry, I don't know much about the discussion, but it works, Mike! thank you
> Mike, Camm!
>
> There is one more question -- Is it possible to define foreign C function
> that has returned value or parameters of user-defined type?
>
> For example, in CLISP (the following stuff is from
> <http://clisp.sourceforge.net/impnotes.html#dffi-examples>):
>
> ======== [ C code ========
> typedef struct {
> int quot; /* Quotient */
> int rem; /* Remainder */
> } div_t;
>
> extern div_t div (int numer, int denom);
> ======== ] C code ========
>
> Translates to:
>
> ======== [ CLISP code ========
> (def-c-struct (div_t :typedef)
> (quot int)
> (rem int))
>
> (def-call-out div (:arguments (numer int) (denom int))
> (:return-type div_t))
>
> (div 20 3) ; Call the function
> #S(DIV_T :QUOT 6 :REM 2) ; returned value of the function
> ======== ] CLISP code ========
>
> Does GCL has similar mechanism to allow such a C function to be defined? or
> some other method achieve the same goal?
> Any relative documents or suggestions? thank you
>


The defined C types at present appear from the docs to be limited to:

C-type:
{ object | int | char | float | double }

Check out gcl-si.{texi,info}. But you can achieve the same by passing
the struct object you want returned as an argument, and setting it
within the function. E.g:

STSET(object,(V95),8, Ct);
^^^^ struct element type
^^^^ struct object (passed as argument)
^^ offset of element in struct
^^ value to set to.

Then just defstruct in lisp whatever struct you want.

This can be improved substantially, but there is no time at the
moment. If you have detailed suggestions or can volunteer a suggested
patch, that would be great!

Take care,

> p.s. SDK stands for Software Development Kit. A set of tools and libraries
> for creating software applications for Windows operating systems.
>
>
> _____________________________________________________
>
> Reply to this item at:
>
> <http://savannah.gnu.org/support/?func=detailitem&item_id=104164>
>
> _____________________________________________
> Message sent via/by Savannah
> http://savannah.gnu.org/
>
>
>
>


--
Camm Maguire -unavailable-
==========================================================================
"The earth is but one country, and mankind its citizens." -- Baha'u'llah

Camm Maguire <camm>
Project Administrator
Thu 05 May 2005 12:36:12 PM UTC, comment #5:

Sorry, I don't know much about the discussion, but it works, Mike! thank you Mike, Camm!

There is one more question -- Is it possible to define foreign C function that has returned value or parameters of user-defined type?

For example, in CLISP (the following stuff is from <http://clisp.sourceforge.net/impnotes.html#dffi-examples>):

======== [ C code ========
typedef struct {
int quot; /* Quotient */
int rem; /* Remainder */
} div_t;

extern div_t div (int numer, int denom);
======== ] C code ========

Translates to:

======== [ CLISP code ========
(def-c-struct (div_t :typedef)
(quot int)
(rem int))

(def-call-out div (:arguments (numer int) (denom int))
(:return-type div_t))

(div 20 3) ; Call the function
#S(DIV_T :QUOT 6 :REM 2) ; returned value of the function
======== ] CLISP code ========

Does GCL has similar mechanism to allow such a C function to be defined? or some other method achieve the same goal?
Any relative documents or suggestions? thank you

p.s. SDK stands for Software Development Kit. A set of tools and libraries for creating software applications for Windows operating systems.

Joe Lee <joe9404>
Wed 04 May 2005 10:23:39 PM UTC, comment #4:

Hi Camm and Joe.

Thanks Camm for the advice - it worked.

Joe, to recap on what Camm says the following two commands achieve what you wanted with the GCL 2.6.6 release package:

(compile-file
"c:/test/sdk-hello.lsp"
:system-p t)

(compiler::link '("c:/test/sdk-hello.o") "c:/test-hello.exe")

Then just double click on "c:/test-hello.exe" and type in your new message box function:

(sdk-hello)

I had a lot of windows open and the message box opened up underneath them, so you may need to search for it as I had to.

The file sdk-hello.lsp was as follows:

======================================================
(clines "#include <windows.h>")
(defentry sdk-hello (object) (object "sdk_hello"))
(defCfun "static object sdk_hello( object ret )"
0
"MessageBox( NULL, \"Hello world!\", \"Hello demo\", MB_OK );"
"return ret;")
======================================================

Interestingly, I didn't need to specify the user32 library so the MessageBox symbol must have been in the GCL image somewhere. I wouldn't expect the entire Win32 API to be available just yet.

This was quite enlightening Joe, so thanks very much for your question and please stay in touch with us!

Cheers

Mike Thomas.

Mike Thomas <mjthomas>
Project Member
Wed 04 May 2005 01:49:34 PM UTC, comment #3:

Greetings!

"Mike Thomas" <mike.thomas@brisbane.paradigmgeo.com> writes:

> | The relevant gcc version is 3.4.2 and insertion of a prototype
> | for sdk_hello just before L1 fixes that problem. Unfortunately
> | the missing initialisation function problem remains in HEAD -
> | perhaps I've called compiler::link incorrectly?.
>
>
> Going further into this it looks like I should use the following command:
>
> >(compiler::link nil "c:/test-hello.exe" "(load
> \"c:/cvs/stable/gcl-2.6.7pre/sdk-hello.o\")" "-luser32" t )
>


The previous call was correct. This one will merely link in user32
with ld, and then load sdk-hello.o in the new image, but ld will omit
all symbols not used at link time by the image, which in this case is
all of them. Which brings up an idea that might be of use in
improving these calls as discussed in my last post -- there is likely
a flag to ld to preserve all symbols, which could be of use in
generating these new relocation sections.

> which does build test-hello.exe but which still has a missing MessageBox
> symbol at load time because the executable only holds onto the symbols in
> user32 which are directly used.
>
> If I have got the command right and I understand the problem, this suggests
> that for the time being you will need to build GCL from scratch with a
> binding for the various Win32 libraries (such as user32) so that those
> symbols are available when GCL is built and hence at object load time. If
> you look at the way I did the JAPI binding you will see what I mean.
>


This is possible, but should not be necessary. Your previous call
with either :system-p t added to compile-file or setting
compiler::default-system-p to non-nil before compiling, should do
the trick.

> The way to fix this is to have some way of searching and selectively
> autoloading from a list of dlls at load time as required by the incoming
> object files (such as, in this case sdk-hello.o).
>


When you get a moment (this is far less urgent than other stuff), your
thoughts on the link/unexec idea I proposed in the last post would be
much appreciated. Perhaps if I get a working example on Linux, this
might focus the discussion. I won't be able to work on thsi for a
while yet.

Take care,

>
>
>
>
>


--
Camm Maguire -unavailable-
==========================================================================
"The earth is but one country, and mankind its citizens." -- Baha'u'llah

Camm Maguire <camm>
Project Administrator
Wed 04 May 2005 01:42:11 PM UTC, comment #2:

Greetigns!

"Mike Thomas" <mike.thomas@brisbane.paradigmgeo.com> writes:

> Camm:
>
> The relevant gcc version is 3.4.2 and insertion of a prototype for sdk_hello
> just before L1 fixes that problem. Unfortunately the missing initialisation
> function problem remains in HEAD - perhaps I've called compiler::link
> incorrectly?.
>
> Cheers
>
> Mike Thomas.
>
> -----Original Message-----
> From: Mike Thomas [mailto:savannah-bounces@gnu.org]
> Sent: Wednesday, 4 May 2005 12:57 PM
> To: camm@enhanced.com; Joe Lee; Mike Thomas;
> -unavailable-
> Subject: [sr #104164] Is it possible to call Windows SDK from GCL?
>
>
>
> Follow-up Comment #1, sr #104164 (project gcl):
>
> Hi Joe.
>
> Thanks for the question, the answer to which exposes a couple of bugs in GCL
> which I hope to sort out for the upcoming new release of GCL - 2.6.7. I'll
> need to discuss these with Camm Maguire as he is better at this stuff that
> me.
>
> The short answer to your question is to use a function called COMPILER-LINK
> which statically links a new GCL image with the new function into a new
> executable.
>


This is indeed the short answer -- you can link in any new symbols you
want that are not present in the original image via compiler::link.
See the documentation for this function in gcl-si.{texi,info}. What
this essentially does is build a new gcl image using a fresh call to
the system linker ld to modify the image symbol table with whatever
new code or libraries you specify. It has the disadvantage that the
image is 'fresh' -- i.e. any work you may have done in the running
image before compiling compiler::link is lost. Such work can be
respecified to run by hand in the fresh new image through one of
compiler::link's arguments, but this is a bit awkward to use.

What I'd like to do in 2.7.0 is to allow the user to link in new
dynamic libraries at runtime, modify the running image's symbol table,
and allow this work to be preserved across image saves via
si::save-system. What this essentially means is that unexec needs to
add a little section to the end of the dumped image containing
relocation records for the new symbols for use by the system's dynamic
linker (ld.so on Linux), i.e. a GOT (Global Access Table). With the
recently completed work on alpha and mips native relocations, I know
how to do this for our bfd platforms, but not yet on Windows and
MaCOSX. Hopefully our GCL experts in these areas might be able to
shed light on this question at some point in the medium term future.

A kludgy way of doing this without any special executable format
knowledge might be to expand the explicit C plt table (mplt in plt.c)
with a bunch of dummy entries referring to unused symbols in external
shared libs, one of which we might provide for this purpose. Then
when new symbols are needed, these symbols could be changed. This
would still require us to be able to find the symbol in the image's
symbol table, but at least we would not have to add any new sections,
etc. Of course this approach is rather awkward and limited too.

> You then run that executable and call your function.
>
> Unfortunately on GCL 2.6.6 this leads to another undefined symbol apparently
> needed for GCL's internal administration:
>
> ============================================================
> >(compile-file "c:/cvs/stable/gcl-2.6.7pre/sdk-hello.lsp")
>
> Compiling c:/cvs/stable/gcl-2.6.7pre/sdk-hello.lsp.
> End of Pass 1.
> End of Pass 2.
> OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3
> Finished compiling c:/cvs/stable/gcl-2.6.7pre/sdk-hello.lsp.
> #p"c:/cvs/stable/gcl-2.6.7pre/sdk-hello.o"
>
> >(compiler::link (list "c:/cvs/stable/gcl-2.6.7pre/sdk-hello.o"
> "libuser32.a") "
> c:/msg.exe")
> ./user-init.o(.text+0x60):user-init.c: undefined reference to
> `init_sdk_hello'
> ./user-init.o(.data+0x0):user-init.c: undefined reference to
> `init_sdk_hello'
> 'c:/raw_msg.exe' is not recognized as an internal or external command,
> operable program or batch file.
>


To use compiler::link, you have to set the :system-p t flag in
compiler-file, or alternately (setq compiler::default-system-p t).
This will ensure that the init functions for each module will be
uniquely named based on the lisp filename, as compiler::link expects.
This should be mentioned in the docs. Otherwise your call above is
fine.

What is the sdk btw?

> Error in LET* [or a callee]: Cannot delete the file #p"c:/raw_msg.exe".
>
> Fast links are on: do (use-fast-links nil) for debugging
> Broken at DELETE-FILE. Type :H for Help.
> 1 (Abort) Return to top level.
> dbl:>>1
>
> Top level.
> ============================================================
>
> I tried this with the CVS HEAD version of GCL and got a gcc type error as
> shown further below.
>
> Camm, the relevant C code was as follows:
>
> static void L1()
> { object *old_base=vs_base;
> object x;
> x=
> sdk_hello(
> vs_base[0]);
> vs_top=(vs_base=old_base)+1;
> vs_base[0]=x;
> }
> / C function defined by DEFCFUN /
>
> static object sdk_hello( object ret )
> {
> object *vs=vs_top;
> object *old_top=vs_top+0;
> {
> MessageBox( NULL, "Hello world!", "Hello demo", MB_OK );
> return ret;
> }
> vs_top=vs;
> }
>
>
> We'll get back to you when we have sorted these problems out.
>
> Cheers
>
> Mike Thomas.
>
> ============================================================
>
> >(compile-file "c:/cvs/stable/gcl-2.6.7pre/sdk-hello.lsp")
>
> Compiling c:/cvs/stable/gcl-2.6.7pre/sdk-hello.lsp.
> End of Pass 1.
> End of Pass 2.
> c:/cvs/stable/gcl-2.6.7pre/sdk-hello.c:5289: error: conflicting types for
> 'sdk_hello'
> c:/cvs/stable/gcl-2.6.7pre/sdk-hello.c:5281: error: previous implicit
> declaration of 'sdk_hello' was here
> (SYSTEM "gcc -c -Wall -DVOL=volatile -fsigned-char -pipe
> -fno-zero-initialized-i
> n-bss -mms-bitfields -march=i686 -mfpmath=387 -Ic:/cvs/head/gcl/unixport/../
> h
> -
> O3 -c -w \"c:/cvs/stable/gcl-2.6.7pre/sdk-hello.c\" -o
> \"c:/cvs/stable/gcl-2.6.
> 7pre/sdk-hello.o\"") returned a non-zero value 1.
>
> Fast links are on: do (use-fast-links nil) for debugging
> Broken at UNLESS.
> 1 (Continue) Continues anyway.
> 2 Return to break level 1.
> 3 Return to top level.
> dbl:>>>


Mike, two questions here:

1) This is 2.6.7pre, not CVS head, no?

2) What are lines 5289 and 5281 (with a little context) of
sdk-hello.c?

>
> _____________________________________________________
>
> Reply to this item at:
>
> <http://savannah.gnu.org/support/?func=detailitem&item_id=104164>
>
> _____________________________________________
> Message sent via/by Savannah
> http://savannah.gnu.org/
>
>
>
>
>
>


--
Camm Maguire -unavailable-
==========================================================================
"The earth is but one country, and mankind its citizens." -- Baha'u'llah

Camm Maguire <camm>
Project Administrator
Wed 04 May 2005 02:56:42 AM UTC, comment #1:

Hi Joe.

Thanks for the question, the answer to which exposes a couple of bugs in GCL which I hope to sort out for the upcoming new release of GCL - 2.6.7. I'll need to discuss these with Camm Maguire as he is better at this stuff that me.

The short answer to your question is to use a function called COMPILER-LINK which statically links a new GCL image with the new function into a new executable.

You then run that executable and call your function.

Unfortunately on GCL 2.6.6 this leads to another undefined symbol apparently needed for GCL's internal administration:

============================================================

>(compile-file "c:/cvs/stable/gcl-2.6.7pre/sdk-hello.lsp")


Compiling c:/cvs/stable/gcl-2.6.7pre/sdk-hello.lsp.
End of Pass 1.
End of Pass 2.
OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3
Finished compiling c:/cvs/stable/gcl-2.6.7pre/sdk-hello.lsp.
#p"c:/cvs/stable/gcl-2.6.7pre/sdk-hello.o"

>(compiler::link (list "c:/cvs/stable/gcl-2.6.7pre/sdk-hello.o" "libuser32.a") "

c:/msg.exe")
./user-init.o(.text+0x60):user-init.c: undefined reference to `init_sdk_hello'
./user-init.o(.data+0x0):user-init.c: undefined reference to `init_sdk_hello'
'c:/raw_msg.exe' is not recognized as an internal or external command,
operable program or batch file.

Error in LET* [or a callee]: Cannot delete the file #p"c:/raw_msg.exe".

Fast links are on: do (use-fast-links nil) for debugging
Broken at DELETE-FILE. Type :H for Help.
1 (Abort) Return to top level.
dbl:>>1

Top level.
============================================================

I tried this with the CVS HEAD version of GCL and got a gcc type error as shown further below.

Camm, the relevant C code was as follows:

static void L1()
{ object *old_base=vs_base;
object x;
x=
sdk_hello(
vs_base[0]);
vs_top=(vs_base=old_base)+1;
vs_base[0]=x;
}
/ C function defined by DEFCFUN /

static object sdk_hello( object ret )
{
object *vs=vs_top;
object *old_top=vs_top+0;
{
MessageBox( NULL, "Hello world!", "Hello demo", MB_OK );
return ret;
}
vs_top=vs;
}

We'll get back to you when we have sorted these problems out.

Cheers

Mike Thomas.

============================================================

>(compile-file "c:/cvs/stable/gcl-2.6.7pre/sdk-hello.lsp")


Compiling c:/cvs/stable/gcl-2.6.7pre/sdk-hello.lsp.
End of Pass 1.
End of Pass 2.
c:/cvs/stable/gcl-2.6.7pre/sdk-hello.c:5289: error: conflicting types for 'sdk_hello'
c:/cvs/stable/gcl-2.6.7pre/sdk-hello.c:5281: error: previous implicit declaration of 'sdk_hello' was here
(SYSTEM "gcc -c -Wall -DVOL=volatile -fsigned-char -pipe -fno-zero-initialized-i
n-bss -mms-bitfields -march=i686 -mfpmath=387 -Ic:/cvs/head/gcl/unixport/../h -
O3 -c -w \"c:/cvs/stable/gcl-2.6.7pre/sdk-hello.c\" -o \"c:/cvs/stable/gcl-2.6.
7pre/sdk-hello.o\"") returned a non-zero value 1.

Fast links are on: do (use-fast-links nil) for debugging
Broken at UNLESS.
1 (Continue) Continues anyway.
2 Return to break level 1.
3 Return to top level.
dbl:>>>

Mike Thomas <mjthomas>
Project Member
Tue 03 May 2005 02:28:13 PM UTC, original submission:

I made a file like this:
---- call-sdk.lsp begin ----
(clines "#include <windows.h>")
(defentry sdk-hello (object) (object "sdk_hello"))
(defCfun "static object sdk_hello( object ret )" 0 "MessageBox( NULL, \"Hello world!\", \"Hello demo\", MB_OK );" "return ret;")
---- call-sdk.lsp end ----

then I ran GCL and tried to call the newly definded function `sdk-hello', but something happened:
---- enter GCL ----

>(compile-file "c:/test/call-sdk.lsp")


Compiling c:/test/call-sdk.lsp.
End of Pass 1.
End of Pass 2.
OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3
Finished compiling c:/test/call-sdk.lsp.
#p"c:/test/call-sdk.o"

>(load "c:/test/call-sdk.o")


Loading c:/test/call-sdk.o
undefined _MessageBoxA@16 symbolstart address -T 10542e20 Finished loading c:/te
st/call-sdk.o
144

>(sdk-hello 'param)


Error in EVAL [or a callee]: Caught fatal error [memory may be damaged]

Fast links are on: do (use-fast-links nil) for debugging
Broken at SDK-HELLO. Type :H for Help.
1 (Abort) Return to top level.
dbl:>>
---- exit CCL ----

GCL told me _MessageBoxA@16 is undefined...how to solve this issue? or some other better suggestions? thank you

p.s. I'm a typical Windows user and new to GCL, MinGW, Linux... etc.
The GCL I got is from this place: <ftp://ftp.gnu.org/pub/gnu/gcl/binaries/stable/gcl_2.6.6.mingw32_ansi_japi_20050210.exe>

Joe Lee <joe9404>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by camm (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 21 Nov 2013 09:34:14 PM UTCcammStatusNone=>Done
      Open/ClosedOpen=>Closed
    Wed 04 May 2005 02:56:42 AM UTCmjthomasCarbon-Copy-=>Added -unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup