Updating bug status based on comment #15, still confirmed with the current development version.

I've had a look at this and it is going to be much harder than I anticipated. I think a full rewrite of Octave's interface with PCRE is required.

Better, I'd say:
>> te = regexp ('abc', 'abc(z?)', 'tokenExtents');
>> celldisp (te)
te{1} =
4 3
>>
>> te = regexp ('abce', '((a)(z)?b(c)(z*))(z)?(e)?', 'tokenExtents');
>> celldisp (te)
te{1} =
1 3
4 3
4 4
>>
>> [T, TE, NM] = regexp (sprintf('%s','John Davis\nRogers, James'), '(?<first>\w+)\s+(?<last>\w+)(?<last>\w+),\s+(?<first>\w+)', 'tokens', 'tokenExtents', 'names');
>>
>> celldisp (T)
T{1}{1} =
John
T{1}{2} =
Davis
T{2}{1} =
nRogers
T{2}{2} =
James
>> celldisp (TE)
TE{1} =
1 4
6 10
TE{2} =
12 18
21 25
>> NM(1).first
ans =
John
>> NM(1).last
ans =
Davis
>> NM(2).first
ans =
James
>> NM(2).last
ans =
nRogers
>>

Okay, at least I got 1 out of 5 answers.
Let's try the following:

Matlab is being a bit opaque, as usual:
>> version
ans =
7.9.0.529 (R2009b)
>> te = regexp ('abc', 'abc(z)?', 'tokenExtents');
>> typeinfo (te)
??? Undefined function or method 'typeinfo' for input arguments of type 'cell'.
>> celldisp (te)
te{1} =
4 3
>> te{:}
ans =
4 3
>> te[:]
??? te[:]

Error: Unbalanced or unexpected parenthesis or bracket.
Here is the result of "help disp" in Matlab:
>> help disp
DISP Display array.
DISP(X) displays the array, without printing the array name. In
all other ways it's the same as leaving the semicolon off an
expression except that empty arrays don't display.
If X is a string, the text is displayed.
=====================
This not quite the way disp() works in Octave, or actually
the way it seems to work in Matlab.

This is infuriating. How does one see the results of a calculation in Matlab? I don't understand why they are making it so difficult to print a data structure.
Let's try just one regexp until we can get it working.

Oops. I cut and pasted from email.
Here is try from savannah
(with TE{:} at the end  did not help)
>> [T, TE, NM] = regexp (sprintf('%s','John Davis\nRogers, James'), '(?<first>\w+)\s+(?<last>\w+)(?<last>\w+),\s+(?<first>\w+)', 'tokens', 'tokenExtents', 'names');
>> disp (T)
{1x2 cell} {1x2 cell}
>> disp (TE)
[2x2 double] [2x2 double]
>> disp (NM)
1x2 struct array with fields:
last
first
>>
>> te = regexp ('abce', '((a)(z)?b(c)(z*))(z)?(e)?', 'tokenExtents');
>> disp (te)
[3x2 double]
>>
>> [S, E, TE, M, T, NM, SP] = regexp ('abc', '(?<all>(a)(?<miss1>z?)b(?<hit1>c)(z*))')
S =
[]
E =
[]
TE =
{}
M =
{}
T =
{}
NM =
0x0 struct array with fields:
all
SP =
'abc'
>>
>> disp (TE)
>> disp (T)
>> disp (NM)
>>
>> TE{:}
>>

Michael,
Are you getting the test code from an email, or cutting and pasting from the verbatim block on Savannah? The verbatim block in the bug report has lines that are longer than 80 characters. In the samples you ran it appears that Matlab is seeing only 80 characters at a time followed by a newline which is leading to bad input messages.
Maybe TE{:} will display the actual contents of the tokenExtents?

Maybe this more explicit code will work.
well, does not look like it:
>> te = regexp ('abc', 'abc(z?)', 'tokenExtents');
>> disp (te)
[1x2 double]
>>
>> te = regexp ('abc', 'abc(z)?', 'tokenExtents');
>> disp (te)
[1x2 double]
>>
>> [T, TE, NM] = regexp (sprintf('%s','John Davis\nRogers, James'),
??? [T, TE, NM] = regexp (sprintf('%s','John Davis\nRogers, James'),

Error: Expression or statement is incorrectpossibly unbalanced (, {, or [.
>> '(?<first>\w+)\s+(?<last>\w+)(?<last>\w+),\s+(?<first>\w+)', 'tokens',
ans =
(?<first>\w+)\s+(?<last>\w+)(?<last>\w+),\s+(?<first>\w+)
ans =
tokens
>> 'tokenExtents', 'names');
??? 'tokenExtents', 'names');

Error: Unbalanced or unexpected parenthesis or bracket.
>> disp (T)
??? Undefined function or variable 'T'.
>> disp (TE)
??? Undefined function or variable 'TE'.
>> disp (NM)
??? Undefined function or variable 'NM'.
>>
>> te = regexp ('abce', '((a)(z)?b(c)(z*))(z)?(e)?', 'tokenExtents');
>> disp (te)
[3x2 double]
>>
>> [S, E, TE, M, T, NM, SP] = regexp ('abc',
??? [S, E, TE, M, T, NM, SP] = regexp ('abc',

Error: Expression or statement is incorrectpossibly unbalanced (, {, or [.
>> '(?<all>(a)(?<miss1>z?)b(?<hit1>c)(z*))')
??? '(?<all>(a)(?<miss1>z?)b(?<hit1>c)(z*))')

Error: Unbalanced or unexpected parenthesis or bracket.
>>
>> disp (TE)
??? Undefined function or variable 'TE'.
>> disp (T)
??? Undefined function or variable 'T'.
>> disp (NM)
??? Undefined function or variable 'NM'.
>>
>>

Thanks Michael. I guess I don't know how to write Matlab code very well because I was expecting that it would actually print the matrix of tokenExtents rather than just declaring it to be a matrix.
Maybe this more explicit code will work.

>> regexp ('abc', 'abc(z?)', 'tokenExtents')
ans =
[1x2 double]
>>
>> regexp ('abc', 'abc(z)?', 'tokenExtents')
ans =
[1x2 double]
>>
>> version
ans =
7.9.0.529 (R2009b)
>> t = regexp (sprintf('%s','John Davis\nRogers, James'), '(?<f.
??? t = regexp (sprintf('%s','John Davis\nRogers, James'), '(?<f.

Error: A MATLAB string constant is not terminated properly.

A fifth test I need run to see how Matlab is handling patterns which don't match in an alternation operator.

I have one more set of regexp tests to run if you have the time.

I have an implementation that works for tokens. Getting the tokenExtents to work correctly is more difficult. Could you run the following corner cases and tell me what Matlab returns?

MATLAB (2011b):
>> [S, E, TE, M, T, NM, SP] = regexp ('bar2','^(foo)?bar(\d*)$')
S =
1
E =
4
TE =
[2x2 double]
M =
'bar2'
T =
{1x2 cell}
NM =
1x1 struct array with no fields.
SP =
'' ''
>> TE{:}
ans =
1 0
4 4
>> T{:}
ans =
'' '2'
>> [S, E, TE, M, T, NM, SP] = regexp ('bar2','^bar(z)?(\d*)$')
S =
1
E =
4
TE =
[2x2 double]
M =
'bar2'
T =
{1x2 cell}
NM =
1x1 struct array with no fields.
SP =
'' ''
>> TE{:}
ans =
4 3
4 4
>> T{:}
ans =
'' '2'

This is confirmed on the development code branch as well.
For reference, could you try the following code bits on Matlab and post what it returns? I'm curious about how they handle token extents for matched but empty substrings which is where the issue lies.

Parenthesized subexpressions create tokens. If the subexpression matches an empty string, an empty string should be returned. Now they just disappear: If I have an expression with two parenthesized subexpressions and the first match empty, only the second token is returned. This is inconsistent with other regex implementations like perl and MATLAB.
This works:
t=regexp('1bar2','^(\d*)bar(\d*)$','tokens');
isequal(t{1}{2},'2')
This doesn't:
t=regexp('bar2','^(\d*)bar(\d*)$','tokens');
isequal(t{1}{2},'2')
t=regexp('bar2','^(foo)?bar(\d*)$','tokens');
isequal(t{1}{2},'2')
