bugGNU Octave - Bugs: bug #54216, Additional items for .hgignore

 
 

bug #54216: Additional items for .hgignore

Submitter:  Andrew Janke <apjanke>
Submitted:  Sun 01 Jul 2018 12:08:15 AM UTC
   
 
Category:  Configuration and Build System Severity:  3 - Normal
Priority:  2 Item Group:  Feature Request
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 31 Dec 2018 03:28:45 PM UTC, comment #23: 

I reverted my changes.  Most of the module.mk for that directory is protected by an AMCOND for building the documentation.  The file macros.texi is needed regardless of whether documentation is being built and trying to extract all the rules and move them outside of the conditional is a big make-work project.

Rik <rik5>
Group administrator
Mon 31 Dec 2018 04:08:30 AM UTC, comment #22: 

I noticed this bug today and did a little bit more work on it.  macros.texi is now a constructed file from macros.txi.  See https://hg.savannah.gnu.org/hgweb/octave/rev/d0c070f5c67a.

Changing octave.texi to be a built file did not appear easy.  I made some inital assays and got all sorts of problems with the build system.  I agree that what we have now is good enough.

Rik <rik5>
Group administrator
Mon 31 Dec 2018 12:36:08 AM UTC, comment #21: 

I don't think we need to keep this bug report open any longer, it's as good as it's going to get. The only improvement would be to explicitly list which .texi files to ignore in .hgignore, which would mean maintaining a list of .txi files in both the makefiles and .hgignore, lots of unnecessary busy work.

Mike Miller <mtmiller>
Group Member
Mon 02 Jul 2018 08:52:09 PM UTC, comment #20: 

Seems fine to change all .texi files to be generated from .txi files, but also doesn't seem strictly necessary and a low priority item.

Mercurial understands what to do.  If we start tracking a file, it will show up as (C)lean or (M)odified, but never (I)gnored or (?)nknown no matter what the rule in .hgignore is.

The only issue I see is that you add a file, myfile.texi, while experimenting.  This will show up as ignored, rather than unknown.

Rik <rik5>
Group administrator
Mon 02 Jul 2018 08:45:06 PM UTC, comment #19: 

I started to look at generating all the .texi files and immediately found that automake fails when it can't find the octave.texi file.  So it needs to be generated by the bootstrap script.  I guess this problem could be solved but it seems complicated.

Maybe it would be easier and more reliable to extend the .hgignore syntax to be able to allow us to say "ignore this pattern except for these files"...

Or just explicitly list all the generated .texi files.

Or, hey, generate the .hgignore file using a script that can have any kind of logic that you need to use!  :-)

John W. Eaton <jwe>
Group administrator
Mon 02 Jul 2018 08:28:24 PM UTC, comment #18: 

It would be OK with me to generate all the .texi files from corresponding .txi files and to use the same method for all, even if some files don't have @DOCSTRING tags.

As for the other issue I raised, I don't think there is a need to keep this report open for that.  I'm fine with ignoring it.

John W. Eaton <jwe>
Group administrator
Mon 02 Jul 2018 07:42:02 PM UTC, comment #17: 

I made an additional change to .hgignore to use the '(^|/)' prefix on any newly ignored files that may be in the source tree or in a build subdirectory (e.g. build-aux/check-subst-vars.sh or mybuilddir/build-aux/check-subst-vars.sh).

I understand what you are saying about the .texi file extension being relatively rare compared to the .txi file extension, but I still believe we should not have a broad pattern that masks valid tracked or potentially newly created file names.

Maybe we should rename macros.texi and octave.texi to have a .txi extension, and have the build system "generate" the corresponding .texi file with cp? That way we can say universally that all .texi files should be ignored.

Mike Miller <mtmiller>
Group Member
Mon 02 Jul 2018 07:18:25 PM UTC, comment #16: 

Should probably re-title the bug report so that it references the remaining issue you want to address.

Rik <rik5>
Group administrator
Mon 02 Jul 2018 07:11:01 PM UTC, comment #15: 

OK, at least there is a warning.  It's probably good enough.

Just for the record, to do this job completely we would need to know whether we are building from hg sources or a tarball.

If a tarball, then make distclean should be sufficient.

If from hg sources, then we would want to check for other generated files that are not removed by the distclean target and that would generated in the build tree.

This might be nice to do, but it does have a really low priority for me.


John W. Eaton <jwe>
Group administrator
Mon 02 Jul 2018 07:07:17 PM UTC, comment #14: 

Mercurial checks whether the file is source controlled before using the ignore file.

There are two files, macros.texi and octave.texi which are source controlled.  Try using 'hg stat' on them.


~/wip/Projects_Mine/octave-dev/doc/interpreter: hg stat -c macros.texi
C macros.texi
~/wip/Projects_Mine/octave-dev/doc/interpreter: hg stat -i macros.texi
~/wip/Projects_Mine/octave-dev/doc/interpreter: hg stat -u macros.texi

~/wip/Projects_Mine/octave-dev/doc/interpreter: hg stat -c plot.texi
~/wip/Projects_Mine/octave-dev/doc/interpreter: hg stat -u plot.texi
~/wip/Projects_Mine/octave-dev/doc/interpreter: hg stat -i plot.texi
I plot.texi




Rik <rik5>
Group administrator
Mon 02 Jul 2018 06:34:11 PM UTC, comment #13: 

This pattern is not good, because it covers some legitimate files that are tracked in hg


+^doc/interpreter/.*\.texi$


Mike Miller <mtmiller>
Group Member
Mon 02 Jul 2018 06:28:06 PM UTC, comment #12: 

I added comments and re-organized the patterns within the .hgignore file.  I also added many, many more patterns.  See http://hg.savannah.gnu.org/hgweb/octave/rev/a27ae22eb41d.

It's still not perfect, but good enough for now.  Closing report

Rik <rik5>
Group administrator
Mon 02 Jul 2018 05:03:16 PM UTC, comment #11: 

Keep in mind that Autoconf already has a standard check that prevents a user from doing something like this


$ ./configure
$ make  ## or not
$ mkdir foo
$ cd foo
$ ../configure
...
configure: error: source directory already configured; run "make distclean" there first


I agree that distclean does not clean up all of our generated files, but it may be good enough for your concerns.

Mike Miller <mtmiller>
Group Member
Mon 02 Jul 2018 04:31:43 PM UTC, comment #10: 

I'm thinking of what might happen if someone does a build in the source tree and then attempts a separate build outside the source tree, possibly without cleaning up all the generated files in the source tree.

Even if this doesn't cause trouble for the rules in the Makefile, it could cause confusion because of two copies of generated files.  If it's not too difficult, I think it would be good to warn about this situation and recommend a command to use to clean the source tree before running bootstrap, configure, and make in a separate build tree.

John W. Eaton <jwe>
Group administrator
Mon 02 Jul 2018 04:18:14 PM UTC, comment #9: 

I see no problem with adding more generated source files and build byproducts to the .hgignore file if you don't mind producing that list.

I don't follow comment #8. Do you mean a user has done an in-source build, followed by a distclean, followed by an out-of-tree build, with leftover generated files in-tree? I'm not sure we can prevent that, and I'm not sure it will be a problem, since the second configure run should cause the timestamps to be out of date and new files will be generated in the build tree. The originals in the source tree should be ignored. Not tested, just speculating.

Mike Miller <mtmiller>
Group Member
Mon 02 Jul 2018 04:12:09 PM UTC, comment #8: 

If we continue to allow building in the source tree, then maybe we should also have some check to ensure that the source tree is free of these generated files that could cause trouble when the build is done outside of the source tree?

John W. Eaton <jwe>
Group administrator
Mon 02 Jul 2018 04:06:21 PM UTC, comment #7: 

I pushed the first bit of the patch from Andrew here (http://hg.savannah.gnu.org/hgweb/octave/rev/975450679c15).

Waiting for feedback on comment #6 before making any further changes or closing this report.

Rik <rik5>
Group administrator
Mon 02 Jul 2018 03:59:09 PM UTC, comment #6: 

I have no problem committing this.  In fact, I always build in the source tree.  It may not be recommended, but plenty of people do it so I offer myself up as a guinea pig to test that configuration (and because I'm lazy).

There are plenty of other things we could also add.  I see that we try to ignore things in build-aux


 ^build-aux/missing$
 ^build-aux/move-if-change$


but there are a whole bunch of other scripts in that directory that are not currently excluded


? build-aux/check-subst-vars.sh
? build-aux/subst-config-vals.sh
? build-aux/subst-cross-config-vals.sh
? build-aux/subst-default-vals.sh
? build-aux/subst-script-vals.sh


What about dirstamps?  Should those be ignored?


? libgui/graphics/.dirstamp
? libgui/graphics/.octave-dirstamp


How about Qt files that are generated by moc in the same way that configure or make might take xxx.in and transform it to xxx?


? libgui/graphics/moc-Backend.cc
? libgui/graphics/moc-ButtonControl.cc
? libgui/graphics/moc-ButtonGroup.cc
? libgui/graphics/moc-ContextMenu.cc
? libgui/graphics/moc-EditControl.cc
? libgui/graphics/moc-Figure.cc
? libgui/graphics/moc-FigureWindow.cc
? libgui/graphics/moc-ListBoxControl.cc
? libgui/graphics/moc-Menu.cc
? libgui/graphics/moc-MouseModeActionGroup.cc
? libgui/graphics/moc-Object.cc
? libgui/graphics/moc-ObjectFactory.cc
? libgui/graphics/moc-ObjectProxy.cc
? libgui/graphics/moc-PopupMenuControl.cc
? libgui/graphics/moc-PushTool.cc
? libgui/graphics/moc-SliderControl.cc
? libgui/graphics/moc-TextEdit.cc
? libgui/graphics/moc-ToggleTool.cc
? libgui/graphics/moc-ToolBar.cc
? libgui/graphics/moc-annotation-dialog.cc


We ignore some results of autoconf, such as ^configure$, but what about these?


? config.cache
? config.h
? config.log
? config.status



Rik <rik5>
Group administrator
Sun 01 Jul 2018 06:50:35 AM UTC, comment #5: 


> In case you aren't aware, the recommended way to build Octave is to create an empty build directory, and build with...


I've switched to doing an out-of-tree build as you suggest, and this is keeping my source tree clean. Nothing new shows up under 'hg status'. Thanks!

Andrew Janke <apjanke>
Sun 01 Jul 2018 01:33:59 AM UTC, comment #4: 

Thanks! I wasn't aware of `make maintainer-clean`. That worked.

Andrew Janke <apjanke>
Sun 01 Jul 2018 12:58:36 AM UTC, comment #3: 

Also please try 'make maintainer-clean' to get your source tree to a truly pristine state. That should delete the 'libinterp/corefcn/*-opts.cc' files for example.

The 'make distclean' target returns the source tree to the state of the distributed source tarball, which contains more files than the hg repository.

Mike Miller <mtmiller>
Group Member
Sun 01 Jul 2018 12:57:48 AM UTC, comment #2: 

That's fair. I'll adjust my personal hgignore to take care of the IDEA stuff.

I'm posting a new 'hgignore-v2.patch' that removes the cmake and IDEA files, leaving just .Plo, .Po, and HG-ID, with HG-ID adjusted to '^HG-ID$'.

Andrew Janke <apjanke>
Sun 01 Jul 2018 12:50:47 AM UTC, comment #1: 

I don't know about others, but my opinion is that the project hgignore file should include files patterns that are created directly by the project build system. Any files that are created by a user's favorite editor or IDE should be in the user's own personal hgignore file.

So I have no problem adding hgignore rules for HG-ID and .Plo and .Po, but the cmake and idea files should not be included.

The HG-ID pattern should be '^HG-ID$' since the file only exists at the top level.

In case you aren't aware, the recommended way to build Octave is to create an empty build directory, and build with


$ ../configure  # (or /path/to/configure)
$ make
$ make check
$ make install


With this approach, most of the build products are not mixed with the source directories and 'hg status' does not report as many untracked files.

Mike Miller <mtmiller>
Group Member
Sun 01 Jul 2018 12:08:15 AM UTC, original submission:  

Hi, Octave folks,

Would you consider adding a few more items to .hgignore? Patch attached.

This includes items that I found in my working directory after doing a build from source, some of which stuck around even after a "make distclean". It also includes an ignore for the JetBrains CLion IDE, which drops a project file into the source tree.


^CMakeLists.txt
^cmake-build-debug
HG-ID

.*\.Plo
.*\.Po

# IDEs
\.idea


Also, after doing a `./configure` and `make`, and then a `make distclean`, I still have a few untracked files sitting around in my working copy. These look like they were generated as part of the build. How should these be handled?


$ hg status
? libgui/mk-default-qt-settings.sh
? libinterp/corefcn/DASPK-opts.cc
? libinterp/corefcn/DASRT-opts.cc
? libinterp/corefcn/DASSL-opts.cc
? libinterp/corefcn/LSODE-opts.cc
? libinterp/corefcn/Quad-opts.cc
? libinterp/corefcn/oct-tex-parser.yy


Andrew Janke <apjanke>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #44470:  hgignore-v2.patch added by apjanke (688B - application/octet-stream)
file #44469:  hgignore.patch added by apjanke (769B - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by apjanke (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 13 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-12-31 mtmiller Carbon-CopyRemoved 80942 -
    2018-12-31 mtmiller StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2018-07-04 rik5 Priority5 - Normal 2
    2018-07-02 jwe StatusFixed In Progress
        Open/ClosedClosed Open
    2018-07-02 rik5 StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2018-07-02 rik5 Operating SystemMac OS Any
    2018-07-02 rik5 StatusPatch Submitted In Progress
    2018-07-01 mtmiller StatusNone Patch Submitted
    2018-07-01 apjanke Attached File- Added hgignore-v2.patch, #44470
    2018-07-01 apjanke Attached File- Added hgignore.patch, #44469

    Back to the top

    Powered by Savane 3.13-cf05.
    Corresponding source code