bugThe GNU Hurd - Bugs: bug #29809, creating a firmlink loop results...

 
 

bug #29809: creating a firmlink loop results in memory exhaustion

Submitter:  Justus Winter <teythoon>
Submitted:  Thu 06 May 2010 05:09:09 PM UTC
   
 
Category:  Hurd Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  None Privacy:  Public
Assigned to:  None Originator Name: 
Open/Closed:  Open Reproducibility:  Every Time
Size (loc):  None Planned Release:  None
Effort:  0.00
Wiki-like text discussion box: 


* Mandatory Fields

Add a New Comment Rich Markup
   

Wed 07 Sep 2016 07:43:08 PM UTC, comment #3: 

Perhaps this could be fixed by adding three parameters to the dir_lookup routine:

  • in max_total_lookups = Total number of directory lookups that may be caused by this call.  If it needs more, then fail with ELOOP.  If the server thinks the value of max_total_lookups is unreasonably high, then it can use a smaller value instead.
  • out total_lookups = Total number of directory lookups that the dir_lookup call caused before succeeding.  If the caller is calling dir_lookup in a loop, it can add total_lookups to a cumulative total and then compare the sum to its own max_total_lookups limit.
  • in max_lookup_depth = Number of recursion levels permitted during this call.  If it needs more, then fail with ELOOP.  If the server thinks the value of max_lookup_depth is unreasonably high, then it can use a smaller value instead.


Then these would also have to be added to the various functions in <hurd/lookup.h>, and the translators would have to be changed to propagate this infomation.  I think it would be straightforward to implement and pretty easy to understand, and it would reliably give the desired ELOOP result.  On the other hand, it would need changes in many source files, and it would not prevent someone from hacking a custom translator to deliberately fake the values, but then again such a program can flood the system with threads and RPCs anyway.

I was also thinking about an alternative solution: implement migrating threads and set a quota on the number of activations per thread; make RPCs fail if they would exceed the quota. However, that scheme would let users deliberately consume almost all of the activation quota of a thread and then call a filesystem server, which would attempt to call a storage server and fail. It might be difficult for the filesystem server to recover from such errors.

Kalle Olavi Niemitalo <kon>
Wed 12 May 2010 11:58:29 AM UTC, comment #2: 

Here is what happens with a settrans -ac foo; ls foo:

  1. Client does a dir_lookup("foo") on CWD, presumably on ext2fs
  2. ext2fs does a fsys_getroot() on firmlink
  3. firmlink does a dir_lookup("foo") on (same) CWD
  4. Back to (2)


The important bit is that control never goes back to the client, which
would've made glibc fail with ELOOP after a certain number of retries.
The core problem is related to a poor reaction to resource exhaustion,
but we might want to add loop detection to firmlink as a special case
if it can be done easily.

There is no way to the differentiate the first fsys_getroot() call from
the subsequent ones, so this can't be the base for loop detection.

So can we give the client back control so it can detect loops?  No,
because having access to any retry could lead to privilege escalation,
and any subsequent magical or reauth retry would also be interpreted in
the wrong context.  This really puts a nail in the coffin because firmlink
is itself a client to all the translators between it and the target.
So it seems it is impossible to detect arbitrary loops, at least, not
without changing the lookup protocol somehow.

However, tight loops can be detected by opening the target node with
O_NOLINK and check whether the result is the symlink-like port returned
by firmlink for such requests.  This would be easy enough, and I think
doing the check at startup would do the trick and cheap enough to be
worth it for even if it is limited to tight loops.

Carl Fredrik Hammar <hammy>
Group Member
Fri 07 May 2010 01:13:45 AM UTC, comment #1: 

As suggested by antrik in #hurd here is a log of the port usage of both ext2fs and firmlink (both eat up more and more memory) measured once every second using portinfo $PID | wc --lines:

root@hurdbox:~# ps ax | egrep '(ext2fs|firmlink)'
root         3  - Ro    0:05.51 ext2fs --readonly --multiboot-command-line=root
root       345  - R     0:00.47 /hurd/firmlink foo
root       348 p0 S     0:00.01 egrep (ext2fs|firmlink)
root@hurdbox:~# ./port_count 3 345
  0: 23276 22956
  1: 27901 27541
  2: 32182 31846
  3: 36311 36066
  4: 40491 40276
  5: 44646 44376
  6: 48641 48461
  7: 52716 52501
  8: 56652 56461
  9: 60601 60426
 10: 64521 64341
 11: 68506 68356
 12: 72436 72346
 13: 76449 76386

Justus Winter <teythoon>
Group Member
Thu 06 May 2010 05:09:09 PM UTC, original submission:  

cd /tmp ; touch bla ; settrans -f bla /hurd/firmlink bla ; ls bla

  • wait some time, play around with vmstat or the like
  • and the pager will start to flood the console with


(default pager): dropping data_request because of previous paging errors

Justus Winter <teythoon>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by kon (Posted a comment)
  • -email is unavailable- added by hammy (Posted a comment)
  • -email is unavailable- added by teythoon (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code