bugThe GNU Hurd - Bugs: bug #29809, creating a firmlink loop results...

 
 

bug #29809: creating a firmlink loop results in memory exhaustion

Submitted by:  Justus Winter <teythoon>
Submitted on:  Thu May 6 17:09:09 2010  
 
Category: HurdSeverity: 3 - Normal
Priority: 5 - NormalItem Group: None
Status: NonePrivacy: Public
Assigned to: NoneOriginator Name: 
Open/Closed: OpenReproducibility: Every Time
Size (loc): NonePlanned Release: None
Effort: 0.00
Wiki-like text discussion box: 

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Wed Sep 7 19:43:08 2016, comment #3:

Perhaps this could be fixed by adding three parameters to the dir_lookup routine:

  • in max_total_lookups = Total number of directory lookups that may be caused by this call. If it needs more, then fail with ELOOP. If the server thinks the value of max_total_lookups is unreasonably high, then it can use a smaller value instead.
  • out total_lookups = Total number of directory lookups that the dir_lookup call caused before succeeding. If the caller is calling dir_lookup in a loop, it can add total_lookups to a cumulative total and then compare the sum to its own max_total_lookups limit.
  • in max_lookup_depth = Number of recursion levels permitted during this call. If it needs more, then fail with ELOOP. If the server thinks the value of max_lookup_depth is unreasonably high, then it can use a smaller value instead.

Then these would also have to be added to the various functions in <hurd/lookup.h>, and the translators would have to be changed to propagate this infomation. I think it would be straightforward to implement and pretty easy to understand, and it would reliably give the desired ELOOP result. On the other hand, it would need changes in many source files, and it would not prevent someone from hacking a custom translator to deliberately fake the values, but then again such a program can flood the system with threads and RPCs anyway.

I was also thinking about an alternative solution: implement migrating threads and set a quota on the number of activations per thread; make RPCs fail if they would exceed the quota. However, that scheme would let users deliberately consume almost all of the activation quota of a thread and then call a filesystem server, which would attempt to call a storage server and fail. It might be difficult for the filesystem server to recover from such errors.

Kalle Olavi Niemitalo <kon>
Wed May 12 11:58:29 2010, comment #2:

Here is what happens with a settrans -ac foo; ls foo:

  1. Client does a dir_lookup("foo") on CWD, presumably on ext2fs
  2. ext2fs does a fsys_getroot() on firmlink
  3. firmlink does a dir_lookup("foo") on (same) CWD
  4. Back to (2)

The important bit is that control never goes back to the client, which
would've made glibc fail with ELOOP after a certain number of retries.
The core problem is related to a poor reaction to resource exhaustion,
but we might want to add loop detection to firmlink as a special case
if it can be done easily.

There is no way to the differentiate the first fsys_getroot() call from
the subsequent ones, so this can't be the base for loop detection.

So can we give the client back control so it can detect loops? No,
because having access to any retry could lead to privilege escalation,
and any subsequent magical or reauth retry would also be interpreted in
the wrong context. This really puts a nail in the coffin because firmlink
is itself a client to all the translators between it and the target.
So it seems it is impossible to detect arbitrary loops, at least, not
without changing the lookup protocol somehow.

However, tight loops can be detected by opening the target node with
O_NOLINK and check whether the result is the symlink-like port returned
by firmlink for such requests. This would be easy enough, and I think
doing the check at startup would do the trick and cheap enough to be
worth it for even if it is limited to tight loops.

Carl Fredrik Hammar <hammy>
Project Member
Fri May 7 01:13:45 2010, comment #1:

As suggested by antrik in #hurd here is a log of the port usage of both ext2fs and firmlink (both eat up more and more memory) measured once every second using portinfo $PID | wc --lines:

root@hurdbox:~# ps ax | egrep '(ext2fs|firmlink)'
root 3 - Ro 0:05.51 ext2fs --readonly --multiboot-command-line=root
root 345 - R 0:00.47 /hurd/firmlink foo
root 348 p0 S 0:00.01 egrep (ext2fs|firmlink)
root@hurdbox:~# ./port_count 3 345
0: 23276 22956
1: 27901 27541
2: 32182 31846
3: 36311 36066
4: 40491 40276
5: 44646 44376
6: 48641 48461
7: 52716 52501
8: 56652 56461
9: 60601 60426
10: 64521 64341
11: 68506 68356
12: 72436 72346
13: 76449 76386

Justus Winter <teythoon>
Project Member
Thu May 6 17:09:09 2010, original submission:

cd /tmp ; touch bla ; settrans -f bla /hurd/firmlink bla ; ls bla

  • wait some time, play around with vmstat or the like
  • and the pager will start to flood the console with

(default pager): dropping data_request because of previous paging errors

Justus Winter <teythoon>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by kon (Posted a comment)
  • -unavailable- added by hammy (Posted a comment)
  • -unavailable- added by teythoon (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    No Changes Have Been Made to This Item

    Back to the top


    Powered by Savane 3.1-cleanup