bugmake - Bugs: bug #51309, Determination of a file list from...

 
 

bug #51309: Determination of a file list from a single folder without changing the working directory

Submitted by:  Markus Elfring <elfring>
Submitted on:  Sun 25 Jun 2017 05:53:27 PM UTC  
 
Severity: 3 - NormalItem Group: Enhancement
Status: Wont FixPrivacy: Public
Assigned to: NoneOpen/Closed: Closed
Component Version: 4.2.1Operating System: Any
Fixed Release: NoneTriage Status: None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Thu 20 Jul 2017 09:04:36 AM UTC, comment #10:

Would you like to compare run time characteristics a bit more with an approach similar to “$(shell find ~/Projekte/Bau/OCamlbuild -name '*.txt' -printf "%f ")”?

Markus Elfring <elfring>
Sun 02 Jul 2017 08:14:31 PM UTC, comment #9:

I suggested also a comparison of two algorithms.

  • A simple time measurement can also show test results for which you might not be looking for.
  • The usage of data structures like list and trees has got also different run time characteristics. Each of them can have a corresponding application domain.
  • Which test environments would you find representative?

> From this I conclude that the performance impact of an additional notdir function is negligible and there's no performance advantage to avoiding it.


It might matter more under other circumstances. (The current software can be fast enough to some degree.)

Markus Elfring <elfring>
Sun 02 Jul 2017 07:45:35 PM UTC, comment #8:

If, for example, you can show an example of notdir adding significantly to the amount of time the make program takes.

To test this I created a makefile like this:

where the "..." represents 6000 paths hardcoded into the makefile. I then ran "time make":

Then I modified the makefile to add a notdir like this:

where the "..." is the same as last time. I got this result:

Same time. This means that the amount of time needed to run notdir on 6000 paths is less than 1/1000th of a second. From this I conclude that the performance impact of an additional notdir function is negligible and there's no performance advantage to avoiding it.

Paul D. Smith <psmith>
Project Administrator
Sun 02 Jul 2017 04:42:13 PM UTC, comment #7:

> (a) you prefer to do it this way rather than the currently supported method of post-processing the wildcard results, …


Yes.

>(b) there's a significant performance benefit to avoiding the extra processing.


I find that the difference can be eventually measured.

We have got different opinions about the circumstances when it would be significant enough.

> … adding new features in a backward- and POSIX-compatible way is difficult due …


How many software developers would dare to extend corresponding programming interfaces so that they can be better reused by the make function library?

I am curious if this story could ever turn into a more promising “maybe” thing.

Markus Elfring <elfring>
Sun 02 Jul 2017 04:31:59 PM UTC, comment #6:

A theoretical argument is not what Paul meant by "actual results". Have you measured the time spent adding and removing the directory names in some realistic case? If not, then please do not advocate for making Make bigger..

Martin Dorey <mdorey>
Sun 02 Jul 2017 04:24:29 PM UTC, comment #5:

I understand what you are suggesting.

But I don't see any need to add new capabilities for this when the existing features of GNU make can already give these results.

The only reasons for it that I can see would be (a) you prefer to do it this way rather than the currently supported method of post-processing the wildcard results, or (b) there's a significant performance benefit to avoiding the extra processing. For (a), I'm not inclined to create new capabilities just for this reason. For (b), I seriously doubt that there will be any measurable performance difference in a real makefile.

If you can show an example where there is a significant performance difference then we can discuss that.

To be clear, my attitudes on makefile syntax are that (1) it's already complex and there are already a lot of features, complexities, and "gotchas", (2) adding new features in a backward- and POSIX-compatible way is difficult due to the relatively free-form syntax of makefiles, and (3) features combine in ways that are multiplicative, not additive, which greatly increases the potential for incorrect interactions, bugs, and the testing needed to support them.

Thus, my initial answer to adding new capabilities is almost always going to be "no". It requires a clear improvement in power (not just syntactic sugar) and a reasonable syntax that makes sense and doesn't break lots of existing makefiles to clear that hurdle and get to "maybe" or "yes".

Paul D. Smith <psmith>
Project Administrator
Sun 02 Jul 2017 03:53:17 PM UTC, comment #4:

I repeat my argument from 26. June:
https://lists.gnu.org/archive/html/bug-make/2017-06/msg00198.html

If the desired directory on which a file filter should be applied could be selected before, the file names would not contain references to this directory and another function call is therefore unnecessary to remove it from the returned list elements.

Markus Elfring <elfring>
Sun 02 Jul 2017 03:38:03 PM UTC, comment #3:

There's no question that

is more efficient than:

because the latter involves starting a whole new shell (two, technically) while the former is handled entirely within the make process with no fork/exec operation required.

If you can show actual results where adding the notdir makes a significant performance difference in a makefile environment we can reconsider, but I'm not interested in adding new features duplicating existing capabilities without proof that there's a real need.

Paul D. Smith <psmith>
Project Administrator
Sun 02 Jul 2017 03:27:53 PM UTC, comment #2:

I disagree with the run time characteristics of the mentioned functionality for my special use case.

Can a similar data processing operation be performed faster by advanced make functions?

Markus Elfring <elfring>
Sun 02 Jul 2017 03:08:44 PM UTC, comment #1:

The current capabilities (using notdir to remove the entire directory, or using patsubst etc. to remove partial directories) is sufficient. Nothing new is needed here.

Paul D. Smith <psmith>
Project Administrator
Sun 25 Jun 2017 05:53:27 PM UTC, original submission:

The function “wildcard” is supported by the make software.
Corresponding file patterns work also in combination with directory names. Such combinations are occasionally useful. The query result will contain them as usual.

I came along the need to query a single folder without changing the working directory so that only file names (without folder names) will be in the result.
Can another software extension become useful for the function library to make such data processing more efficient?

Markus Elfring <elfring>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mdorey (Posted a comment)
  • -unavailable- added by psmith (Posted a comment)
  • -unavailable- added by elfring (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun 02 Jul 2017 03:08:44 PM UTCpsmithStatusNone=>Wont Fix
      Open/ClosedOpen=>Closed

    Back to the top


    Powered by Savane 3.1-cleanup1