Date: 8 Jan 90 Message No: 022
Resent: 11 Jan 90
To: TeX implementors and distributors
From: Barbara Beeton
Subject: Updates: files at labrea, TEX82.BUG, Knuth off-line
[ The initial mailing of this message and the TEX82.BUG file, in three
parts, caused our mailer to suffer a serious breakdown; not only were
my messages not delivered, but the entire mail server crashed, putting
the whole Society out of the mail business for two days. In order to
recover in the most painless manner possible, I have made the recommended
changes in the mailing list, and am re-sending the whole lot. Please
accept my apologies if you get two copies of anything. The only change
to any of them is the addition of the Resent: date at the top of this
message, and this paragraph. ]
Preparing this installment has taken rather longer than I intended,
delayed by standards meetings, the holidays, and writing the user
documentation for the impending release of version 2.0 of both AMS-TeX
and the AMSFonts collection. It is my understanding that the AMS
management wishes these two packages to have the widest distribution
possible, so an announcement should be forthcoming to make them both
available to TeX distributors and archives for inclusion in their
offerings. (The AMSFonts package will not include the .mf sources;
those will be available only by individual request, and there wil be
some restrictions. If you are interesed in the .mf sources, write to
Regina Girouard, rmg@math.ams.com; she is Manager of the AMS Composition
Services Department, in charge of the font distribution.) Stay tuned.
There have been no updates to the TeX "core" at labrea.stanford.edu
since November. On the assumption that TUGboat 10 #4 (the Proceedings
of this summer's tenth anniversary meeting) would be out and in your
hands by now, I had not sent out the updates to the TEX82.BUG file,
since it is very large, and all entries since the beginning of 1989
are printed in their entirety in a supplement accompanying that issue.
However, since there have been a number of delays in its production (it
was sent to the printer the first week of November), and the projected
mailing date now isn't until late January, I am sending under separate
cover, in two pieces, the additions that haven't been sent out before
in this form, items 355-370. (They "duplicate" the code in the TEX.WEB
differences file that was sent early in November, as a supplement to
message 20.) Since preparing the camera copy for the TUGboat supplement,
two additional bug listings, 371 and 372, have come to my attention.
371 is in response to a bug found by Wayne Sullivan, and Knuth distributed
372 in TeXhax 89 #110. Both are reproduced below. The TEX.WEB file
incorporating them has not yet appeared at labrea, as mentioned earlier.
This is the commentary that came with 371 (dated 24 Nov 89); it sounds
like caution is in order when applying it:
Here's how I plan to fix the problem. The code isn't tested yet --- I'll
wait a couple weeks to see if I can combine it with some other work ---
but I think it catches all cases where scanning can interfere with
the save_stack protocols. It preserves the locality of csnaming, and
makes things like \discretionary consistent with \insert in this respect.
And here is the text that accompanied #372; again, caution:
I just received a message from Rainer Sch\"opf and Frank Mittelbach
in Gutenberg's city (Mainz) that they've done it again, found another
subtle bug. This one is especially unique because it will not be
detected in the TRIP test; it only occurs in the system-dependent code
that figures out the "area" and "extension" part of a file name.
As far as I know, the error occurs only when you try to combine
\csname somehow with file name specification; for example,
\input area:file.ext\csname xyz\endcsname
will fail on many systems. (Reason: TeX may have to store the
control sequence name xyz, while it's looking for the token following
"ext". This moves things in the string pool; but the present TeX doesn't
move the area and extension pointers, so they might point to garbage.)
This bug will be corrected in version 2.993, but I decided to use
TeXhax to let installers get a head start, since this bug affects
nearly everybody's system-dependent code! The changes are simple---
so simple I haven't even felt the need to test them yet---but they
should be made, and you can get a head start by putting them into
your change files before you get a new TEX.WEB file.
Here are the changes I'll be making to TEX.WEB (unless there's
an error here, which cannot be:)
Brian Hamilton Kelly pointed out that identifiers in TEX.WEB containing
a cap "E" (in his case, from local changes to support editors) get the
"E" lowered as it is in TeX, and suggested that perhaps this should be
fixed directly in the .WEB file. We'd made a local change for that here
at AMS some years ago, so I forwarded the code to Brian. I also sent
it to Don Knuth, who replied that he didn't think it appropriate in the
canonical TEX.WEB, but suggested that I forward the code to this list.
The solution isn't perfect -- it only suppresses the problem if the "E"
doesn't precede an "X" -- but it's a start. If anyone has a better
solution, please send it to me and i'll distribute it to the list.
The code is given below.
Peter Breitenlohner has provided a list of modules in MF.WEB 1.8 that
have changed, as compared to 1.7; see below. I have not yet retrieved
the source from labrea so that a difference list can be run, but it's
on my list.
Finally, it's now official -- Knuth has withdrawn from the electronic
networks, and will henceforth be accessible only via the postal service.
However, he has deputized me to keep an eye out for genuine bugs (as
reported through TeXhax, UKTeX, etc.) and has agreed to let his secretary
accept electronic messages from me. Mail sent to Knuth's old address at
Sail will now yield the following result:
Subject: Failed mail returned: user not accepting mail
The message below was not deliverable to:
DEK@SAIL.Stanford.EDU
DEK (Don Knuth) has decided to discontinue reading email.
Please write to him at the following address:
Donald E. Knuth
Professor of The Art of Computer Programming
Computer Science Department
Stanford University
Stanford, CA 94305-2140 USA
If you have a bug to report and you would like me to forward it, I will.
Alternatively, you can send your report on paper to Knuth's new address.
########################################################################
Addenda to TEX82.BUG -- items 371 and 372
371. Prevent save_stack conflicts, e.g. in {\hbox\expandafter{\csname
\endcsname}} (found by Sullivan).
@x module 645
@p procedure scan_spec; {scans a box specification and left brace}
label found;
begin if scan_keyword("to") then saved(0):=exactly
@.to@>
else if scan_keyword("spread") then saved(0):=additional
@.spread@>
else begin saved(0):=additional; saved(1):=0;
goto found;
end;
scan_normal_dimen; saved(1):=cur_val;
found: save_ptr:=save_ptr+2; scan_left_brace;
@y
@p procedure scan_spec(@!c:group_code;@!three_codes:boolean);
{scans a box specification and left brace}
label found;
var @!s:integer; {temporarily saved value}
@!spec_code:exactly..additional;
begin if three_codes then s:=saved(0);
if scan_keyword("to") then spec_code:=exactly
@.to@>
else if scan_keyword("spread") then spec_code:=additional
@.spread@>
else begin spec_code:=additional; cur_val:=0;
goto found;
end;
scan_normal_dimen;
found: if three_codes then
begin saved(0):=s; incr(save_ptr);
end;
saved(0):=spec_code; saved(1):=cur_val; save_ptr:=save_ptr+2;
new_save_level(c); scan_left_brace;
@z
@x module 774
scan_spec; new_save_level(align_group);@/
@y
scan_spec(align_group,false);@/
@z
@x module 1073
if t=0 then saved(0):=cur_val@+else saved(0):=-cur_val;
scan_box;
end;
any_mode(leader_ship): begin saved(0):=leader_flag-a_leaders+cur_chr; scan_box;
end;
any_mode(make_box): begin saved(0):=0; begin_box;
end;
@y
if t=0 then scan_box(cur_val)@+else scan_box(-cur_val);
end;
any_mode(leader_ship): scan_box(leader_flag-a_leaders+cur_chr);
any_mode(make_box): begin_box(0);
@z
@x module 1075
procedure box_end;
var p:pointer; {|ord_noad| for new box in math mode}
begin if saved(0)<box_flag then @<Append box |cur_box| to the current list,
shifted by |saved(0)|@>
else if saved(0)<ship_out_flag then @<Store \(c)|cur_box| in a box register@>
else if cur_box<>null then
if saved(0)>ship_out_flag then @<Append a new leader node that
uses |cur_box|@>
@y
procedure box_end(@!box_context:integer);
var p:pointer; {|ord_noad| for new box in math mode}
begin if box_context<box_flag then @<Append box |cur_box| to the current list,
shifted by |box_context|@>
else if box_context<ship_out_flag then @<Store \(c)|cur_box| in a box register@>
else if cur_box<>null then
if box_context>ship_out_flag then @<Append a new leader node that
uses |cur_box|@>
@z
@x module 1076
begin shift_amount(cur_box):=saved(0);
@y
begin shift_amount(cur_box):=box_context;
@z
@x module 1077
if saved(0)<box_flag+256 then
eq_define(box_base-box_flag+saved(0),box_ref,cur_box)
else geq_define(box_base-box_flag-256+saved(0),box_ref,cur_box)
@y
if box_context<box_flag+256 then
eq_define(box_base-box_flag+box_context,box_ref,cur_box)
else geq_define(box_base-box_flag-256+box_context,box_ref,cur_box)
@z
@x module 1078
begin append_glue; subtype(tail):=saved(0)-(leader_flag-a_leaders);
@y
begin append_glue; subtype(tail):=box_context-(leader_flag-a_leaders);
@z
@x module 1079
procedure begin_box;
@y
procedure begin_box(@!box_context:integer);
@z
@x ibid
box_end; {in simple cases, we use the box immediately}
@y
box_end(box_context); {in simple cases, we use the box immediately}
@z
@x module 1083
incr(save_ptr); scan_spec;
if k=hmode then
if (saved(-3)<box_flag)and(abs(mode)=vmode) then
new_save_level(adjusted_hbox_group)
else new_save_level(hbox_group)
else begin if k=vmode then new_save_level(vbox_group)
else begin new_save_level(vtop_group); k:=vmode;
@y
saved(0):=box_context;
if k=hmode then
if (box_context<box_flag)and(abs(mode)=vmode) then
scan_spec(adjusted_hbox_group,true)
else scan_spec(hbox_group,true)
else begin if k=vmode then scan_spec(vbox_group,true)
else begin scan_spec(vtop_group,true); k:=vmode;
@z
@x module 1084
procedure scan_box; {the next input should specify a box or perhaps a rule}
begin @<Get the next non-blank non-relax...@>;
if cur_cmd=make_box then begin_box
else if (saved(0)>=leader_flag)and((cur_cmd=hrule)or(cur_cmd=vrule)) then
begin cur_box:=scan_rule_spec; box_end;
@y
procedure scan_box(@!box_context:integer);
{the next input should specify a box or perhaps a rule}
begin @<Get the next non-blank non-relax...@>;
if cur_cmd=make_box then begin_box(box_context)
else if (box_context>=leader_flag)and((cur_cmd=hrule)or(cur_cmd=vrule)) then
begin cur_box:=scan_rule_spec; box_end(box_context);
@z
@x module 1086
pop_nest; box_end;
@y
pop_nest; box_end(saved(0));
@z
@x module 1117
else begin incr(save_ptr); saved(-1):=0; scan_left_brace;
new_save_level(disc_group); push_nest; mode:=-hmode; space_factor:=1000;
@y
else begin incr(save_ptr); saved(-1):=0; new_save_level(disc_group);
scan_left_brace; push_nest; mode:=-hmode; space_factor:=1000;
@z
@x module 1119
incr(saved(-1)); scan_left_brace; new_save_level(disc_group);
@y
incr(saved(-1)); new_save_level(disc_group); scan_left_brace;
@z
@x module 1167
mmode+vcenter: begin scan_spec; new_save_level(vcenter_group); normal_paragraph;
@y
mmode+vcenter: begin scan_spec(vcenter_group,false); normal_paragraph;
@z
@x module 1172
scan_left_brace; push_math(math_choice_group);
@y
push_math(math_choice_group); scan_left_brace;
@z
@x module 1174
incr(saved(-1)); scan_left_brace; push_math(math_choice_group);
@y
incr(saved(-1)); push_math(math_choice_group); scan_left_brace;
@z
@x module 1241
if global then saved(0):=box_flag+256+cur_val
else saved(0):=box_flag+cur_val;
scan_optional_equals; scan_box;
@y
if global then n:=256+cur_val@+else n:=cur_val;
scan_optional_equals; scan_box(box_flag+n);
@z
372. Bugfix 339 didn't go far enough (found by Sch\"opf and Mittelbach).
@x module 516 [NOTE: THIS AFFECTS ALMOST ALL CHANGE FILES!]
@ And here's the second.
@^system dependencies@>
@p function more_name(@!c:ASCII_code):boolean;
begin if c=" " then more_name:=false
else begin if (c=">")or(c=":") then
begin area_delimiter:=pool_ptr; ext_delimiter:=0;
end
else if (c=".")and(ext_delimiter=0) then ext_delimiter:=pool_ptr;
str_room(1); append_char(c); {contribute |c| to the current string}
@y
@ And here's the second. The string pool might change as the file name is
being scanned, since a new \.{\\csname} might be entered; therefore we keep
|area_delimiter| and |ext_delimiter| relative to the beginning of the current
string, instead of assigning an absolute address like |pool_ptr| to them.
@^system dependencies@>
@p function more_name(@!c:ASCII_code):boolean;
begin if c=" " then more_name:=false
else begin str_room(1); append_char(c); {contribute |c| to the current string}
if (c=">")or(c=":") then
begin area_delimiter:=cur_length; ext_delimiter:=0;
end
else if (c=".")and(ext_delimiter=0) then ext_delimiter:=cur_length;
@z
@x module 517 [NOTE: THIS DOES TOO, AND SO DOES THE NEXT!]
else begin cur_area:=str_ptr; incr(str_ptr);
str_start[str_ptr]:=area_delimiter+1;
@y
else begin cur_area:=str_ptr;
str_start[str_ptr+1]:=str_start[str_ptr]+area_delimiter; incr(str_ptr);
@z
@x ibid
else begin cur_name:=str_ptr; incr(str_ptr);
str_start[str_ptr]:=ext_delimiter; cur_ext:=make_string;
@y
else begin cur_name:=str_ptr;
str_start[str_ptr+1]:=str_start[str_ptr]+ext_delimiter-area_delimiter-1;
incr(str_ptr); cur_ext:=make_string;
@z
########################################################################
Change to TEX.WEB (and others) to avoid dropped E in identifiers
@x [0] (AMS) Avoid dropped E in identifiers other than TeX
\def\drop{\kern-.1667em\lower.5ex\hbox{E}\kern-.125em} % middle of TeX
\catcode`E=13 \uppercase{\def E{e}}
\def\\#1{\hbox{\let E=\drop\it#1\/\kern.05em}} % italic type for identifiers
@y
\def\drop{\kern-.1667em\lower.5ex\hbox{E}\kern-.125em} % middle of TeX
\def\cape#1{\if X#1\drop\else E\fi#1} % other E's in identifiers
\catcode`E=13 \uppercase{\def E{e}}
\def\\#1{\hbox{\let E=\cape\it#1\/\kern.05em}} % italic type for identifiers
@z
########################################################################
% List of changed modules in MF.WEB 1.8 (as compared to 1.7)
% Peter Breitenlohner
% The [c]'s refer to the parts of the book `METAFONT: The Program'
limbo
[1] m.2, 11, 14
[2] m.17-19, 21-23
[3] m.30, 34
[4] m.37-38, 41, 45, 47-49
[5] m.59-60, 66
[6] m.85
[10] m.158-159
[12] m.186, 190, 192-193, 198-199
[13] m.200, 210-212
[14] m.223
[21] m.402, 410-412, 415-417, 422, 424-425, 436
[24] m.493
[35] m.717
[38] m.774-775, 784
[41] m.883
[42] m.912-913, 976-977
[45] m.1093, 1096-1097, 1103-1112, 1134-1141
[47] m.1160
[48] m.1192-1193, 1195, 1200
[49] m.1204-1205
########################################################################
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Character code reference
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Upper case letters: ABCDEFGHIJKLMNOPQRSTUVWXYZ
% Lower case letters: abcdefghijklmnopqrstuvwxyz
% Digits: 0123456789
% Square, curly, angle braces, parentheses: [] {} <> ()
% Backslash, slash, vertical bar: \ / |
% Punctuation: . ? ! , : ;
% Underscore, hyphen, equals sign: _ - =
% Quotes--right left double: ' ` "
%"at", "number" "dollar", "percent", "and": @ # $ % &
% "hat", "star", "plus", "tilde": ^ * + ~
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
[ end of message 022 ]
355. String startup problems (Wayne Sullivan, 17 Jul 89)
@x module 31 (Warning: This affects most change files!)
overflow("buffer size",buf_size);
@:TeX capacity exceeded buffer size}{\quad buffer size@>
@y
@<Report overflow of the input buffer, and abort@>;
@z
@x module 35
consist of the remainder of the command line, after the part that invoked
\TeX.
@y
consist of the remainder of the command line, after the part that invoked
\TeX.
The first line is special also because it may be read before \TeX\ has
input a format file. In such cases, normal error messages cannot yet
be given. The following code uses concepts that will be explained later.
@<Report overflow of the input buffer, and abort@>=
if format_ident=0 then
begin write_ln(term_out,'Buffer size exceeded!'); goto final_end;
@.Buffer size exceeded@>
end
else begin cur_input.loc_field:=first; cur_input.limit_field:=last-1;
overflow("buffer size",buf_size);
@:TeX capacity exceeded buffer size}{\quad buffer size@>
end
@z
@x module 1310
k:=pool_ptr-4; undump_four_ASCII
@y
k:=pool_ptr-4; undump_four_ASCII;
init_str_ptr:=str_ptr; init_pool_ptr:=pool_ptr
@z
@x module 1332
tini@/
@y
init_str_ptr:=str_ptr; init_pool_ptr:=pool_ptr; fix_date_and_time;
tini@/
@z
@x module 1332
init_str_ptr:=str_ptr; init_pool_ptr:=pool_ptr;@/
@y
@z
356. Allow integer products to be 31 bits long (Mittelbach, 16 Aug 89)
@x module 105
@p function nx_plus_y(@!n:integer;@!x,@!y:scaled):scaled;
begin if n<0 then
begin negate(x); negate(n);
end;
if n=0 then nx_plus_y:=y
else if ((x<=(@'7777777777-y) div n)and(-x<=(@'7777777777+y) div n)) then
nx_plus_y:=n*x+y
else begin arith_error:=true; nx_plus_y:=0;
end;
end;
@y
@d nx_plus_y(#)==mult_and_add(#,@'7777777777)
@d mult_integers(#)==mult_and_add(#,0,@'17777777777)
@p function mult_and_add(@!n:integer;@!x,@!y,@!max_answer:scaled):scaled;
begin if n<0 then
begin negate(x); negate(n);
end;
if n=0 then mult_and_add:=y
else if ((x<=(max_answer-y) div n)and(-x<=(max_answer+y) div n)) then
mult_and_add:=n*x+y
else begin arith_error:=true; mult_and_add:=0;
end;
end;
@z
@x module 1240
if q=multiply then cur_val:=nx_plus_y(eqtb[l].int,cur_val,0)
@y
if q=multiply then
if p=int_val then cur_val:=mult_integers(eqtb[l].int,cur_val)
else cur_val:=nx_plus_y(eqtb[l].int,cur_val,0)
@z
357. Raise the maximum number of tokens normally shown (pointed out by
John Lavagnino in TeXhax on 19 May 89; corrected on 31 Aug 89).
@x module 295
begin if p<>null then show_token_list(link(p),null,1000);
@y
begin if p<>null then show_token_list(link(p),null,10000000);
@z
@x module 1370 (this part of the change is cosmetic only)
show_token_list(link(def_ref),null,10000000); print_ln;
@y
token_show(def_ref); print_ln;
@z
358. Like 353, but with $$\begingroup\eqno$$ (Mittelbach, 25 Jul 89)
@x module 1140
mmode+eq_no: if privileged then start_eq_no;
@y
mmode+eq_no: if privileged then
if cur_group=math_shift_group then start_eq_no
else off_save;
@z
359. Major extension to 8-bit character input. (These changes, and a few that
follow, were discussed during the 10th TUG meeting and installed during
the week of 10 Sep 1989. They bring the implementation of TeX82 up to date with
international developments and experience, and I believe they mark the
final development of this program into a completely stable system.)
Several changes to the documentation (not the program) are omitted here.
@x module 18
@!ASCII_code=0..127; {seven-bit numbers}
@y
@!ASCII_code=0..255; {eight-bit numbers}
@z
@x module 19
@d last_text_char=127 {ordinal number of the largest element of |text_char|}
@<Local variables for init...@>=
@!i:0..last_text_char;
@y
@d last_text_char=255 {ordinal number of the largest element of |text_char|}
@<Local variables for init...@>=
@!i:integer;
@z
@x module 21
xchr[0]:=' '; xchr[@'177]:=' ';
{ASCII codes 0 and |@'177| do not appear in text}
@y
@z
@x module 23
for i:=1 to @'37 do xchr[i]:=' ';
@y changing ' ' to chr(i) here will allow all 8-bit characters to get in
for i:=0 to @'37 do xchr[i]:=' ';
for i:=@'177 to @'377 do xchr[i]:=' ';
@z
@x module 24
for i:=1 to @'176 do xord[xchr[i]]:=i;
@y
for i:=@'200 to @'377 do xord[xchr[i]]:=i;
for i:=0 to @'176 do xord[xchr[i]]:=i;
@z
@x module 38
@<Types...@>=
@!pool_pointer = 0..pool_size; {for variables that point into |str_pool|}
@!str_number = 0..max_strings; {for variables that point into |str_start|}
@ @<Glob...@>=
@!str_pool:packed array[pool_pointer] of ASCII_code; {the characters}
@y [OK to make si(#)==#-128 and so(#)==#+128 (without parens) in change files]
Some \PASCAL\ compilers won't pack integers into a single byte unless the
integers lie in the range |-128..127|. To accommodate such systems
we access the string pool only via macros that can easily be redefined.
@d si(#) == # {convert from |ASCII_code| to |packed_ASCII_code|}
@d so(#) == # {convert from |packed_ASCII_code| to |ASCII_code|}
@<Types...@>=
@!pool_pointer = 0..pool_size; {for variables that point into |str_pool|}
@!str_number = 0..max_strings; {for variables that point into |str_start|}
@!packed_ASCII_code = 0..255; {elements of |str_pool| array}
@ @<Glob...@>=
@!str_pool:packed array[pool_pointer] of packed_ASCII_code; {the characters}
@z
@x module 42
begin str_pool[pool_ptr]:=#; incr(pool_ptr);
@y
begin str_pool[pool_ptr]:=si(#); incr(pool_ptr);
@z
@x module 45
begin if str_pool[j]<>buffer[k] then
@y
begin if so(str_pool[j])<>buffer[k] then
@z
@x module 47
var k,@!l:0..127; {small indices or counters}
@y
var k,@!l:0..255; {small indices or counters}
@z
@x module 47
@<Make the first 128 strings@>;
@y
@<Make the first 256 strings@>;
@z
@x module 48
@ @<Make the first 128...@>=
for k:=0 to 127 do
begin if (@<Character |k| cannot be printed@>) then
begin append_char("^"); append_char("^");
if k<@'100 then append_char(k+@'100)
else append_char(k-@'100);
@y
@ @d app_lc_hex(#)==l:=#;
if l<10 then append_char(l+"0)@+else append_char(l-10+"a")
@<Make the first 256...@>=
for k:=0 to 255 do
begin if (@<Character |k| cannot be printed@>) then
begin append_char("^"); append_char("^");
if k<@'100 then append_char(k+@'100)
else if k<@'200 then append_char(k-@'100)
else begin app_lc_hex(k div 16); app_lc_hex(k mod 16);
end;
@z
@x module 59
begin print_char(str_pool[j]); incr(j);
@y
begin print_char(so(str_pool[j])); incr(j);
@z
@x modules 59 and 60
else if s<128 then
@y
else if s<256 then
@z
@x module 60
begin print(str_pool[j]); incr(j);
@y
begin print(so(str_pool[j])); incr(j);
@z
@x module 63
if c>=0 then if c<128 then print(c);
@y
if c>=0 then if c<256 then print(c);
@z
@x module 68
@ In certain situations, \TeX\ prints either a standard visible ASCII
character or its hexadecimal ASCII code.
@p procedure print_ASCII(@!c:integer); {prints a character or its code}
begin if (c>=0) and (c<=127) then print(c)
else begin print_char("[");
if c<0 then print_int(c)@+else print_hex(c);
print_char("]");
end;
end;
@y
@ Old versions of \TeX\ needed a procedure called |print_ASCII| whose function
is now subsumed by |print|. We retain the old name here as a possible aid to
future software arch\ae ologists.
@d print_ASCII == print
@z
@x module 69
begin print_char(str_pool[j]); n:=n-v;
end;
if n<=0 then return; {nonpositive input produces no output}
k:=j+2; u:=v div (str_pool[k-1]-"0");
if str_pool[k-1]="2" then
begin k:=k+2; u:=u div (str_pool[k-1]-"0");
end;
if n+u>=v then
begin print_char(str_pool[k]); n:=n+u;
end
else begin j:=j+2; v:=v div (str_pool[j-1]-"0");
@y
begin print_char(so(str_pool[j])); n:=n-v;
end;
if n<=0 then return; {nonpositive input produces no output}
k:=j+2; u:=v div (so(str_pool[k-1])-"0");
if str_pool[k-1]=si("2") then
begin k:=k+2; u:=u div (so(str_pool[k-1])-"0");
end;
if n+u>=v then
begin print_char(so(str_pool[k])); n:=n+u;
end
else begin j:=j+2; v:=v div (so(str_pool[j-1])-"0");
@z
@x module 70
begin print_char(str_pool[j]); incr(j);
@y
begin print_char(so(str_pool[j])); incr(j);
@z
@x module 222
@d single_base=active_base+128 {equivalents of one-letter control sequences}
@d null_cs=single_base+128 {equivalent of \.{\\csname\\endcsname}}
@y
@d single_base=active_base+256 {equivalents of one-character control sequences}
@d null_cs=single_base+256 {equivalent of \.{\\csname\\endcsname}}
@z
@x module 230
{table of 128 command codes (the ``catcodes'')}
@d lc_code_base=cat_code_base+128 {table of 128 lowercase mappings}
@d uc_code_base=lc_code_base+128 {table of 128 uppercase mappings}
@d sf_code_base=uc_code_base+128 {table of 128 spacefactor mappings}
@d math_code_base=sf_code_base+128 {table of 128 math mode mappings}
@d int_base=math_code_base+128 {beginning of region 5}
@y
{table of 256 command codes (the ``catcodes'')}
@d lc_code_base=cat_code_base+256 {table of 256 lowercase mappings}
@d uc_code_base=lc_code_base+256 {table of 256 uppercase mappings}
@d sf_code_base=uc_code_base+256 {table of 256 spacefactor mappings}
@d math_code_base=sf_code_base+256 {table of 256 math mode mappings}
@d int_base=math_code_base+256 {beginning of region 5}
@z
@x module 232
for k:=0 to 127 do
@y
for k:=0 to 255 do
@z
@x module 236
@d del_code_base=count_base+256 {128 delimiter code mappings}
@d dimen_base=del_code_base+128 {beginning of region 6}
@y
@d del_code_base=count_base+256 {256 delimiter code mappings}
@d dimen_base=del_code_base+256 {beginning of region 6}
@z
@x module 240
for k:=0 to 127 do del_code(k):=-1;
@y
for k:=0 to 255 do del_code(k):=-1;
@z
@x module 264
begin if s<128 then cur_val:=s+single_base
@y
begin if s<256 then cur_val:=s+single_base
@z
@x ibid
for j:=0 to l-1 do buffer[j]:=str_pool[k+j];
@y
for j:=0 to l-1 do buffer[j]:=so(str_pool[k+j]);
@z
@x module 289
@d cs_token_flag==@'10000 {amount added to the |eqtb| location in a
token that stands for a control sequence; is a multiple of~256}
@y
@d cs_token_flag==@'7777 {amount added to the |eqtb| location in a
token that stands for a control sequence; is a multiple of~256, less~1}
@z
@x module 293
if (info(p)<0)or(c>127) then print_esc("BAD.")
@y
if info(p)<0 then print_esc("BAD.")
@z
@x module 341
@!cat:0..15; {|cat_code(cur_chr)|, usually}
@y
@!cat:0..15; {|cat_code(cur_chr)|, usually}
@!c,@!cc:ASCII_code; {constituents of a possible expanded code}
@!d:2..3; {number of excess characters in an expanded code}
@z
@x module 352
@<If this |sup_mark| starts a control character...@>=
begin if (cur_chr=buffer[loc])and(loc<limit) then
begin if buffer[loc+1]<@'100 then cur_chr:=buffer[loc+1]+@'100
else cur_chr:=buffer[loc+1]-@'100;
loc:=loc+2; goto reswitch;
end;
@y
@ Notice that a code like \.{\^\^8} becomes \.x if not followed by a hex digit.
@d is_hex(#)==(((#>="0")and(#<="9"))or((#>="a")and(#<="f")))
@d hex_to_cur_chr==
if c<="9" then cur_chr:=c-"0" @+else cur_chr:=c-"a"+10;
if cc<="9" then cur_chr:=16*cur_chr+cc-"0"
else cur_chr:=16*cur_chr+cc-"a"+10
@<If this |sup_mark| starts an expanded character...@>=
begin if cur_chr=buffer[loc] then if loc<limit then
begin c:=buffer[loc+1]; @+if c<@'200 then {yes we have an expanded char}
begin loc:=loc+2;
if is_hex(c) then if loc<=limit then
begin cc:=buffer[loc]; @+if is_hex(cc) then
begin incr(loc); hex_to_cur_chr; goto reswitch;
end;
end;
if c<@'100 then cur_chr:=c+@'100 @+else cur_chr:=c-@'100;
goto reswitch;
end;
end;
@z
@x module 355
begin cur_chr:=buffer[k+1];
if cur_chr<@'100 then buffer[k-1]:=cur_chr+@'100
else buffer[k-1]:=cur_chr-@'100;
limit:=limit-2; first:=first-2;
while k<=limit do
begin buffer[k]:=buffer[k+2]; incr(k);
end;
goto start_cs;
end;
@y
begin c:=buffer[k+1]; @+if c<@'200 then {yes, one is indeed present}
begin d:=2;
if is_hex(c) then @+if k+2<=limit then
begin cc:=buffer[k+2]; @+if is_hex(cc) then incr(d);
end;
if d>2 then
begin hex_to_cur_chr; buffer[k-1]:=cur_chr;
end
else if c<@'100 then buffer[k-1]:=c+@'100
else buffer[k-1]:=c-@'100;
limit:=limit-d; first:=first-d;
while k<=limit do
begin buffer[k]:=buffer[k+d]; incr(k);
end;
goto start_cs;
end;
end;
@z
@x module 360
There is one more branch.
@y
There is one more branch.
@d end_line_char_inactive == (end_line_char<0)or(end_line_char>255)
@z
@x ibid
if (end_line_char<0)or(end_line_char>127) then decr(limit)
@y
if end_line_char_inactive then decr(limit)
@z
@x modules 362 and 483 and 538 and 1337
if (end_line_char<0)or(end_line_char>127) then decr(limit)
@y
if end_line_char_inactive then decr(limit)
@z
@x module 391
if (info(r)>match_token+127)or(info(r)<match_token) then s:=null
@y
if (info(r)>match_token+255)or(info(r)<match_token) then s:=null
@z
@x module 407
((cur_chr=str_pool[k])or(cur_chr=str_pool[k]-"a"+"A")) then
@y
((cur_chr=so(str_pool[k]))or(cur_chr=so(str_pool[k])-"a"+"A")) then
@z
@x module 414
begin scan_seven_bit_int;
@y
begin scan_char_num;
@z
@x module 432
|scan_something_internal|:
@<Declare procedures that scan restricted classes of integers@>=
procedure scan_seven_bit_int;
begin scan_int;
if (cur_val<0)or(cur_val>127) then
begin print_err("Bad character code");
@.Bad character code@>
help2("The numeric code for a character must be between 0 and 127.")@/
("I changed this one to zero."); int_error(cur_val); cur_val:=0;
end;
end;
@y
|scan_something_internal|.
@z (and I'll interchange modules 434 and 435)
@x module 442
if cur_val>127 then
@y
if cur_val>255 then
@z
@x module 464
begin t:=str_pool[k];
@y
begin t:=so(str_pool[k]);
@z
@x module 506 (correction needed twice)
if (cur_cmd>active_char)or(cur_chr>127) then
@y
if (cur_cmd>active_char)or(cur_chr>255) then
@z
@x module 519
for j:=str_start[a] to str_start[a+1]-1 do append_to_name(str_pool[j]);
for j:=str_start[n] to str_start[n+1]-1 do append_to_name(str_pool[j]);
for j:=str_start[e] to str_start[e+1]-1 do append_to_name(str_pool[j]);
@y
for j:=str_start[a] to str_start[a+1]-1 do append_to_name(so(str_pool[j]));
for j:=str_start[n] to str_start[n+1]-1 do append_to_name(so(str_pool[j]));
for j:=str_start[e] to str_start[e+1]-1 do append_to_name(so(str_pool[j]));
@z
@x module 526
if (cur_cmd>active_char)or(cur_chr>127) then
@y
if (cur_cmd>active_char)or(cur_chr>255) then
@z
@x module 603 (change to be made twice)
dvi_out(str_pool[k]);
@y
dvi_out(so(str_pool[k]));
@z
@x module 617
for s:=str_start[str_ptr] to pool_ptr-1 do dvi_out(str_pool[s]);
@y
for s:=str_start[str_ptr] to pool_ptr-1 do dvi_out(so(str_pool[s]));
@z
@x module 766
begin case str_pool[r_type*8+t+magic_offset] of
@y
begin case so(str_pool[r_type*8+t+magic_offset]) of
@z
@x module 780
@d span_code=128 {distinct from any character}
@d cr_code=129 {distinct from |span_code| and from any character}
@y
@d span_code=256 {distinct from any character}
@d cr_code=257 {distinct from |span_code| and from any character}
@z
@x module 815
label done,done1,done2,done3,done4;
@y
label done,done1,done2,done3,done4,continue;
@z
@x module 896
else if (type(s)=kern_node)and(subtype(s)=normal) then c:=128
else if type(s)=whatsit_node then c:=128
else goto done1;
if c<128 then if lc_code(c)<>0 then
if (lc_code(c)=c)or(uc_hyph>0) then goto done2
else goto done1;
s:=link(s);
@y
else if (type(s)=kern_node)and(subtype(s)=normal) then goto continue
else if type(s)=whatsit_node then goto continue
else goto done1;
if lc_code(c)<>0 then
if (lc_code(c)=c)or(uc_hyph>0) then goto done2
else goto done1;
continue: s:=link(s);
@z
@x module 897
if c>=128 then goto done3;
if (lc_code(c)=0)or(hn=63) then goto done3;
hb:=s; incr(hn); hu[hn]:=c; hc[hn]:=lc_code(c)-1;
@y
if lc_code(c)=0 then goto done3;
if hn=63 then goto done3;
hb:=s; incr(hn); hu[hn]:=c; hc[hn]:=lc_code(c);
@z
@x module 898
if c>=128 then goto done3;
if (lc_code(c)=0)or(j=63) then goto done3;
incr(j); hu[j]:=c; hc[j]:=lc_code(c)-1;@/
@y
if lc_code(c)=0 then goto done3;
if j=63 then goto done3;
incr(j); hu[j]:=c; hc[j]:=lc_code(c);@/
@z
@x module 923
hc[0]:=127; hc[hn+1]:=127; hc[hn+2]:=128; {insert delimiters}
@y
hc[0]:=0; hc[hn+1]:=0; hc[hn+2]:=256; {insert delimiters}
@z
@x ibid
while hc[l]=trie_char(z) do
@y
while hc[l]=qo(trie_char(z)) do
@z
@x module 931
repeat if str_pool[u]<hc[j] then goto not_found;
if str_pool[u]>hc[j] then goto done;
@y
repeat if so(str_pool[u])<hc[j] then goto not_found;
if so(str_pool[u])>hc[j] then goto done;
@z
@x module 937
else begin if (cur_chr>127)or(lc_code(cur_chr)=0) then
@y
else begin if lc_code(cur_chr)=0 then
@z
@x ibid
begin incr(n); hc[n]:=lc_code(cur_chr)-1;
@y
begin incr(n); hc[n]:=lc_code(cur_chr);
@z
@x module 945
@!init @!trie_c:packed array[trie_pointer] of ASCII_code; {characters to match}
@y
@!init @!trie_c:packed array[trie_pointer] of packed_ASCII_code;
{characters to match}
@z
% In modules 951--954, change "127" to "255" and "128" to "256"
@x module 952
trie_link(0):=0; trie_char(0):=0; trie_op(0):=min_quarterword;
@y
trie_link(0):=0; trie_char(0):=min_quarterword; trie_op(0):=min_quarterword;
@z
@x module 953
begin c:=trie_c[p];
@y
begin c:=so(trie_c[p]);
@z
@x module 955
begin if trie_link(h+trie_c[q])=0 then goto not_found;
@y
begin if trie_link(h+so(trie_c[q]))=0 then goto not_found;
@z
@x module 956
repeat z:=h+trie_c[q]; trie_back(trie_link(z)):=trie_back(z);
@y
repeat z:=h+so(trie_c[q]); trie_back(trie_link(z)):=trie_back(z);
@z
@x module 959
begin q:=trie_l[p]; c:=trie_c[p];
trie_link(z+c):=trie_ref[q]; trie_char(z+c):=c; trie_op(z+c):=trie_o[p];
@y
begin q:=trie_l[p]; c:=so(trie_c[p]);
trie_link(z+c):=trie_ref[q]; trie_char(z+c):=qi(c); trie_op(z+c):=trie_o[p];
@z
@x module 962
begin if cur_chr="." then cur_chr:=128 {edge-of-word delimiter}
else begin cur_chr:=lc_code(cur_chr);
if cur_chr=0 then
begin print_err("Nonletter");
@.Nonletter@>
help1("(See Appendix H.)"); error; cur_chr:=128;
end;
end;
if k<63 then
begin incr(k); hc[k]:=cur_chr-1; hyf[k]:=0; digit_sensed:=false;
end;
end
else begin hyf[k]:=cur_chr-"0";
if k<63 then digit_sensed:=true;
end
@y
begin if cur_chr="." then cur_chr:=0 {edge-of-word delimiter}
else begin cur_chr:=lc_code(cur_chr);
if cur_chr=0 then
begin print_err("Nonletter");
@.Nonletter@>
help1("(See Appendix H.)"); error;
end;
end;
if k<63 then
begin incr(k); hc[k]:=cur_chr; hyf[k]:=0; digit_sensed:=false;
end;
end
else if k<63 then
begin hyf[k]:=cur_chr-"0"; digit_sensed:=true;
end;
end
@z
@x module 963
while (p>0)and(c>trie_c[p]) do
@y
while (p>0)and(c>so(trie_c[p])) do
@z
@x ibid
if (p=0)or(c<trie_c[p]) then
@y
if (p=0)or(c<so(trie_c[p])) then
@z
@x module 964
trie_c[p]:=c; trie_o[p]:=min_quarterword;
@y
trie_c[p]:=si(c); trie_o[p]:=min_quarterword;
@z
@x module 965
if hc[1]=127 then hyf[0]:=0;
if hc[k]=127 then hyf[k]:=0;
@y
if hc[1]=0 then hyf[0]:=0;
if hc[k]=0 then hyf[k]:=0;
@z
@x module 966
h.rh:=0; h.b0:=min_quarterword; h.b1:=0; {|trie_link:=0|,
|trie_op:=min_quarterword|, |trie_char:=0|}
@y
h.rh:=0; h.b0:=min_quarterword; h.b1:=min_quarterword; {|trie_link:=0|,
|trie_op:=min_quarterword|, |trie_char:=qi(0)|}
@z
@x module 1034
if c<128 then
@y
@z
@x ibid
else space_factor:=1000
@y
@z
@x module 1151
letter,other_char,char_given: if cur_chr>=128 then c:=cur_chr
else begin c:=ho(math_code(cur_chr));
@y
letter,other_char,char_given: begin c:=ho(math_code(cur_chr));
@z
@x module 1154
mmode+letter,mmode+other_char,mmode+char_given: if cur_chr<128 then
set_math_char(ho(math_code(cur_chr)))
else set_math_char(cur_chr);
mmode+char_num: begin scan_char_num; cur_chr:=cur_val;
if cur_chr<128 then set_math_char(ho(math_code(cur_chr)))
else set_math_char(cur_chr);
@y
mmode+letter,mmode+other_char,mmode+char_given:
set_math_char(ho(math_code(cur_chr)));
mmode+char_num: begin scan_char_num; cur_chr:=cur_val;
set_math_char(ho(math_code(cur_chr)));
@z
@x module 1232
p:=cur_chr; scan_seven_bit_int; p:=p+cur_val; scan_optional_equals;
@y
p:=cur_chr; scan_char_num; p:=p+cur_val; scan_optional_equals;
@z
@x module 1233
else n:=127
@y
else n:=255
@z
@x module 1289
begin if t>=cs_token_flag then t:=t-active_base;
c:=t mod 256;
if c<128 then if equiv(b+c)<>0 then t:=256*(t div 256)+equiv(b+c);
if t>=cs_token_flag then info(p):=t+active_base
else info(p):=t;
@y
begin c:=t mod 256;
if equiv(b+c)<>0 then info(p):=t-c+equiv(b+c);
@z
@x module 1309
w.b0:=str_pool[k]; w.b1:=str_pool[k+1];
w.b2:=str_pool[k+2]; w.b3:=str_pool[k+3];
@y [often qi(so(x))=x, but not e.g. when "quarterwords" are two bytes]
w.b0:=qi(so(str_pool[k])); w.b1:=qi(so(str_pool[k+1]));
w.b2:=qi(so(str_pool[k+2])); w.b3:=qi(so(str_pool[k+3]));
@z
@x module 1310
str_pool[k]:=w.b0; str_pool[k+1]:=w.b1;
str_pool[k+2]:=w.b2; str_pool[k+3]:=w.b3
@y
str_pool[k]:=si(qo(w.b0)); str_pool[k+1]:=si(qo(w.b1));
str_pool[k+2]:=si(qo(w.b2)); str_pool[k+3]:=si(qo(w.b3))
@z
@x module 1368
for k:=str_start[str_ptr] to pool_ptr-1 do dvi_out(str_pool[k]);
@y
for k:=str_start[str_ptr] to pool_ptr-1 do dvi_out(so(str_pool[k]));
@z
360. Major change to allow multiple hyphenation tables.
I've combined this with the code for a minor change:
361. New parameters \lefthyphenmin and \righthyphenmin.
@x module 11 (affects most change files!)
@!trie_size=8000; {space for hyphenation patterns; should be larger for
\.{INITEX} than it is in production versions of \TeX}
@y
@!trie_size=8000; {space for hyphenation patterns; should be larger for
\.{INITEX} than it is in production versions of \TeX}
@!trie_op_size=500; {space for ``opcodes'' in the hyphenation patterns}
@z
@x module 212
known as |space_factor|; it holds the current space factor used in spacing
calculations. In math mode, |aux| is also known as |incompleat_noad|; if
@y
known as |space_factor| and |clang|; it holds the current space factor used in
spacing calculations, and the current language used for hyphenation.
(The value of |clang| is undefined in restricted horizontal mode.)
In math mode, |aux| is also known as |incompleat_noad|; if
@z
@x ibid
@!pg_field,@!aux_field,@!ml_field: integer;
@y
@!pg_field,@!ml_field: integer;
@!aux_field: memory_word;
@z
@x module 213
@d prev_depth==aux {the name of |aux| in vertical mode}
@d space_factor==aux {the name of |aux| in horizontal mode}
@d incompleat_noad==aux {the name of |aux| in math mode}
@y
@d prev_depth==aux.sc {the name of |aux| in vertical mode}
@d space_factor==aux.hh.lh {part of |aux| in horizontal mode}
@d clang==aux.hh.rh {the other part of |aux| in horizontal mode}
@d incompleat_noad==aux.int {the name of |aux| in math mode}
@z
@x module 218
@!a:integer; {auxiliary}
@y
@!a:memory_word; {auxiliary}
@z
@x module 219
if a<=ignore_depth then print("ignored")
else print_scaled(a);
if nest[p].pg_field<>0 then
begin print(", prevgraf ");
print_int(nest[p].pg_field); print(" line");
if nest[p].pg_field<>1 then print_char("s");
end;
end;
1: begin print_nl("spacefactor "); print_int(a);
end;
2: if a<>null then
begin print("this will be denominator of:"); show_box(a);
@y
if a.sc<=ignore_depth then print("ignored")
else print_scaled(a.sc);
if nest[p].pg_field<>0 then
begin print(", prevgraf ");
print_int(nest[p].pg_field); print(" line");
if nest[p].pg_field<>1 then print_char("s");
end;
end;
1: begin print_nl("spacefactor "); print_int(a.hh.lh);
if m>0 then if a.hh.rh>0 then
begin print(", current language "); print_int(a.hh.rh);
end;
end;
2: if a.int<>null then
begin print("this will be denominator of:"); show_box(a.int);
@z
@x modules 236--238
@d int_pars=50 {total number of integer parameters}
@y
@d language_code=50 {current hyphenation table}
@d left_hyphen_min_code=51 {minimum left hyphenation fragment size}
@d right_hyphen_min_code=52 {minimum right hyphenation fragment size}
@d int_pars=53 {total number of integer parameters}
@z and define/print these new parameters appropriately (mimicking new_line_char)
@x module 418
else begin cur_val:=aux;
if m=vmode then cur_val_level:=dimen_val@+else cur_val_level:=int_val;
end
@y
else if m=vmode then
begin cur_val:=prev_depth; cur_val_level:=dimen_val;
end
else begin cur_val:=space_factor; cur_val_level:=int_val;
end
@z
@x module 775
begin mode:=-vmode; prev_depth:=nest[nest_ptr-2].aux_field;
@y
begin mode:=-vmode; prev_depth:=nest[nest_ptr-2].aux_field.sc;
@z
@x module 786
begin push_nest; mode:=(-hmode-vmode)-mode; aux:=0;
@y
begin push_nest; mode:=(-hmode-vmode)-mode;
if mode=-hmode then space_factor:=0 @+else prev_depth:=0;
@z
@x module 863
loop@+ begin @<Create an active breakpoint representing the beginning of
the paragraph@>;
@y
loop@+ begin if threshold>inf_bad then threshold:=inf_bad;
if second_pass then @<Initialize for hyphenating a paragraph@>;
@<Create an active breakpoint representing the beginning of the paragraph@>;
@z
@x module 891
passes on only about 5 per cent of the paragraphs.
@y
passes on only about 5 per cent of the paragraphs.
@<Initialize for hyphenating...@>=
begin @!init if trie_not_ready then init_trie; @+tini@;@/
l_hyf:=left_hyphen_min-1;@+if l_hyf<0 then l_hyf:=0;
r_hyf:=right_hyphen_min-1;@+if r_hyf<0 then r_hyf:=0;
min_hyf:=l_hyf+r_hyf+2; cur_lang:=0;
end
@z
@x module 892
@!hyf_char:integer; {hyphen character of the relevant font}
@y
@!hyf_char:integer; {hyphen character of the relevant font}
@!cur_lang:ASCII_code; {current hyphenation table of interest}
@!l_hyf,@!r_hyf,@!min_hyf:integer; {limits on fragment sizes}
@z
@x module 894
begin s:=link(cur_p);
@y
begin if min_hyf>63 then goto done1;
s:=link(cur_p);
@z
@x module 896
else if type(s)=whatsit_node then goto continue
@y
else if type(s)=whatsit_node then
begin @<Advance \(p)past a whatsit node in the \(p)pre-hyphenation loop@>;
goto continue;
end
@z
@x module 899 (title of this module also changes appropriately)
if hn<5 then goto done1;
@y
if hn<min_hyf then goto done1;
@z
@x module 902
for j:=2 to hn-3 do if odd(hyf[j]) then goto found1;
@y
for j:=l_hyf+1 to hn-r_hyf-1 do if odd(hyf[j]) then goto found1;
@z
@x module 920
$p_1\ldots p_k$ by setting |@t$z_1$@>:=@t$p_1$@>| and then, for |1<i<=k|,
@y
$p_1\ldots p_k$ by letting $z_0$ be one greater than the relevant language index
and then, for |1<=i<=k|,
@z
@x ibid
the letters in |hc[(l-k+1)..l@,]|, we perform all of the required operations
for this pattern by carrying out the following little program: Set
|v:=trie_op(@t$z_k$@>)|. Then set |hyf[l-hyf_distance[v]]:=@tmax@>(
hyf[l-hyf_distance[v]], hyf_num[v])|, and |v:=hyf_next[v]|; repeat, if
necessary, until |v=min_quarterword|.
@y
the letters in |hc[(l-k+1)..l@,]| of language |t|,
we perform all of the required operations
for this pattern by carrying out the following little program: Set
|v:=trie_op(@t$z_k$@>)|. Then set |v:=v+op_start[t]|,
|hyf[l-hyf_distance[v]]:=@tmax@>(hyf[l-hyf_distance[v]], hyf_num[v])|,
and |v:=hyf_next[v]|; repeat, if necessary, until |v=min_quarterword|.
@z
@x module 921
@!hyf_distance:array[quarterword] of small_number; {position |k-j| of $n_j$}
@!hyf_num:array[quarterword] of small_number; {value of $n_j$}
@!hyf_next:array[quarterword] of quarterword; {continuation of this |trie_op|}
@y
@!hyf_distance:array[1..trie_op_size] of small_number; {position |k-j| of $n_j$}
@!hyf_num:array[1..trie_op_size] of small_number; {value of $n_j$}
@!hyf_next:array[1..trie_op_size] of quarterword; {continuation code}
@!op_start:array[ASCII_code] of 0..trie_op_size; {offset for current language}
@z
@x module 922
@!v:quarterword; {an index into |hyf_distance|, etc.}
@y
@!v:integer; {an index into |hyf_distance|, etc.}
@z
@x module 923
for j:=0 to hn-2 do
begin z:=hc[j]; l:=j;
@y
if trie_char(cur_lang+1)<>qi(cur_lang) then return; {no patterns for |cur_lang|}
for j:=0 to hn-r_hyf do
begin z:=trie_link(cur_lang+1)+hc[j]; l:=j;
@z
@x ibid
found: hyf[1]:=0; hyf[hn-2]:=0; hyf[hn-1]:=0; hyf[hn]:=0
@y
found: for j:=0 to l_hyf do hyf[j]:=0;
for j:=0 to r_hyf do hyf[hn-j]:=0
@z
@x module 924
repeat i:=l-hyf_distance[v];
@y
repeat v:=v+op_start[cur_lang]; i:=l-hyf_distance[v];
@z
@x module 930
h:=hc[1];
@y
h:=hc[1]; incr(hn); hc[hn]:=cur_lang;
@z
@x ibid
not_found:
@y
not_found: decr(hn)
@z
@x module 931
goto found;
@y
decr(hn); goto found;
@z
@x module 934
@p procedure new_hyph_exceptions; {enters new exceptions}
label reswitch, exit, found, not_found, done;
@y
@d set_cur_lang==if language<=0 then cur_lang:=0
else if language>255 then cur_lang:=0
else cur_lang:=language
@p procedure new_hyph_exceptions; {enters new exceptions}
label reswitch, exit, found, not_found;
@z
@x ibid
begin scan_left_brace; {a left brace must follow \.{\\hyphenation}}
@y
begin scan_left_brace; {a left brace must follow \.{\\hyphenation}}
set_cur_lang;
@z
@x module 935
spacer,right_brace: begin if n>4 then @<Enter a hyphenation exception@>;
@y
spacer,right_brace: begin if n>1 then @<Enter a hyphenation exception@>;
@z
@x module 938
begin if n>1 then
@y
begin if n<63 then
@z
@x module 939
begin str_room(n); h:=0;
@y
begin incr(n); hc[n]:=cur_lang; str_room(n); h:=0;
@z
@x ibid
loop@+ begin if p=null then goto done;
if info(p)<n-2 then goto done;
q:=link(p); free_avail(p); p:=q; {eliminate hyphens that \TeX\ doesn't like}
end;
done: @<Insert the \(p)pair |(s,p)| into the exception table@>;
@y
@<Insert the \(p)pair |(s,p)| into the exception table@>;
@z
@x module 942
@p@!init @<Declare procedures for preprocessing hyphenation patterns@>@;
@y
@<Declare subprocedures for |line_break|@>=
@!init @<Declare procedures for preprocessing hyphenation patterns@>@;
@z
@x module 943
@d quarterword_diff=max_quarterword-min_quarterword
@d trie_op_hash_size=quarterword_diff+quarterword_diff {double}
@<Glob...@>=
@!init@! trie_op_hash:array[0..trie_op_hash_size] of quarterword;
{trie op codes for triples}
tini@;@/
@t\hskip1em@>@!trie_op_ptr:quarterword; {highest |trie_op| assigned}
@y
@<Glob...@>=
@!init@! trie_op_hash:array[-trie_op_size..trie_op_size] of 0..trie_op_size;
{trie op codes for quadruples}
@!trie_used:array[ASCII_code] of quarterword;
{largest opcode used so far for this language}
@!trie_op_lang:array[1..trie_op_size] of ASCII_code;
{language part of a hashed quadruple}
@!trie_op_val:array[1..trie_op_size] of quarterword;
{opcode corresponding to a hashed quadruple}
@!trie_op_ptr:0..trie_op_size; {number of stored ops so far}
tini
@z
@x module 944 should be entirely replaced
@y by the following code:
@ It's tempting to remove the |overflow| stops in the following procedure;
|new_trie_op| could return |min_quarterword| (thereby simply ignoring
part of a hyphenation pattern) instead of aborting the job. However, that would
lead to different hyphenation results on different installations of \TeX\
using the same patterns. The |overflow| stops are necessary for portability
of patterns.
@<Declare procedures for preprocessing hyph...@>=
function new_trie_op(@!d,@!n:small_number;@!v:quarterword):quarterword;
label exit;
var h:-trie_op_size..trie_op_size; {trial hash location}
@!u:quarterword; {trial op code}
@!l:0..trie_op_size; {pointer to stored data}
begin h:=abs(n+313*d+361*v+1009*cur_lang) mod (trie_op_size+trie_op_size)
- trie_op_size;
loop@+ begin l:=trie_op_hash[h];
if l=0 then {empty position found for a new op}
begin if trie_op_ptr=trie_op_size then
overflow("pattern memory ops",trie_op_size);
u:=trie_used[cur_lang];
if u=max_quarterword then
overflow("pattern memory ops per language",
max_quarterword-min_quarterword);
incr(trie_op_ptr); incr(u); trie_used[cur_lang]:=u;
hyf_distance[trie_op_ptr]:=d;
hyf_num[trie_op_ptr]:=n; hyf_next[trie_op_ptr]:=v;
trie_op_lang[trie_op_ptr]:=cur_lang; trie_op_hash[h]:=trie_op_ptr;
trie_op_val[trie_op_ptr]:=u; new_trie_op:=u; return;
end;
if (hyf_distance[l]=d)and(hyf_num[l]=n)and(hyf_next[l]=v)
and(trie_op_lang[l]=cur_lang) then
begin new_trie_op:=trie_op_val[l]; return;
end;
if h>-trie_op_size then decr(h)@+else h:=trie_op_size;
end;
exit:end;
@z
@x module 945: move the code from this module into 946, and introduce new code:
@y
@ After |new_trie_op| has compressed the necessary opcode information,
plenty of information is available to unscramble the data into the
final form needed by our hyphenation algorithm.
@<Sort the hyphenation op tables into proper order@>=
op_start[0]:=-min_quarterword;
for j:=1 to 255 do op_start[j]:=op_start[j-1]+qo(trie_used[j-1]);
for j:=1 to trie_op_ptr do
trie_op_hash[j]:=op_start[trie_op_lang[j]]+trie_op_val[j]; {destination}
for j:=1 to trie_op_ptr do while trie_op_hash[j]>j do
begin k:=trie_op_hash[j];@/
t:=hyf_distance[k]; hyf_distance[k]:=hyf_distance[j]; hyf_distance[j]:=t;@/
t:=hyf_num[k]; hyf_num[k]:=hyf_num[j]; hyf_num[j]:=t;@/
t:=hyf_next[k]; hyf_next[k]:=hyf_next[j]; hyf_next[j]:=t;@/
trie_op_hash[j]:=trie_op_hash[k]; trie_op_hash[k]:=k;
end
@z
@x module 949: Move this code to just after the new module 945!
mentioned so far, let's write a procedure that does the initialization.
@<Declare procedures for preprocessing hyph...@>=
procedure init_pattern_memory; {gets ready to build a linked trie}
var h:0..trie_op_hash_size; {an index into |trie_op_hash|}
@!p:trie_pointer; {an index into |trie_hash|}
begin for h:=0 to trie_op_hash_size do trie_op_hash[h]:=min_quarterword;
trie_op_ptr:=min_quarterword; trie_root:=0; trie_c[0]:=0; trie_ptr:=0;
for p:=0 to trie_size do trie_hash[p]:=0;
end;
@y
mentioned so far, let's write down the code that gets them started.
@<Initialize table entries...@>=
for k:=-trie_op_size to trie_op_size do trie_op_hash[k]:=0;
for k:=0 to 255 do trie_used[k]:=min_quarterword;
trie_op_ptr:=0;
@z
@x module 950
@t\hskip1em@>@!trie_min:trie_pointer;
{all locations |<=trie_min| are vacant in |trie|}
tini@;@/
@t\hskip1em@>@!trie_max:trie_pointer; {largest location used in |trie|}
@y
@t\hskip10pt@>@!trie_min:array[ASCII_code] of trie_pointer;
{the first possible slot for each character}
@t\hskip10pt@>@!trie_max:trie_pointer; {largest location used in |trie|}
@t\hskip10pt@>@!trie_not_ready:boolean; {is the trie still in linked form?}
tini
@z
@x modules 951 and 952
@ Here is how these data structures are initialized.
@<Declare procedures for preprocessing hyph...@>=
procedure init_trie_memory; {gets ready to pack into |trie|}
var p:trie_pointer; {index into |trie_ref|, |trie|, |trie_taken|}
begin for p:=0 to trie_ptr do trie_ref[p]:=0;
trie_max:=256; trie_min:=256; trie_link(0):=1; trie_taken[0]:=false;
trie_link(trie_size):=0; trie_back(0):=trie_size; {wrap around}
for p:=1 to 256 do
begin trie_back(p):=p-1; trie_link(p):=p+1; trie_taken[p]:=false;
end;
end;
@ Each time \.{\\patterns} appears, it overrides any patterns that were
entered earlier, so the arrays are not initialized until \TeX\ sees
\.{\\patterns}. However, some of the global variables must be
initialized when \.{INITEX} is loaded, in case the user never mentions
any \.{\\patterns}.
@<Initialize table entries...@>=
trie_op_ptr:=min_quarterword;@/
trie_link(0):=0; trie_char(0):=min_quarterword; trie_op(0):=min_quarterword;
for k:=1 to 255 do trie[k]:=trie[0];
trie_max:=255;
@y
@ Each time \.{\\patterns} appears, it contributes further patterns to
the future trie, which will be built only when hyphenation is attempted or
when a format file is dumped. The boolean variable |trie_not_ready|
will change to |false| when the trie is compressed; this will disable
further patterns.
@<Initialize table entries...@>=
trie_not_ready:=true; trie_root:=0; trie_c[0]:=si(0); trie_ptr:=0;
@ Here is how the trie-compression data structures are initialized.
If storage is tight, it would be possible to overlap |trie_op_hash|,
|trie_op_lang|, and |trie_op_val| with |trie|, |trie_hash|, and |trie_taken|,
because we finish with the former just before we need the latter.
@<Get ready to compress the trie@>=
@<Sort the hyphenation...@>;
for p:=0 to trie_size do trie_hash[p]:=0;
trie_root:=compress_trie(trie_root); {identify equivalent subtries}
for p:=0 to trie_ptr do trie_ref[p]:=0;
for p:=0 to 255 do trie_min[p]:=p+1;
trie_link(0):=1; trie_max:=0
@z
@x module 953
begin c:=so(trie_c[p]);
if c<trie_min then trie_min:=c;
if trie_min=0 then z:=trie_link(trie_size)
else z:=trie_link(trie_min-1); {get the first conceivably good hole}
loop@+ begin if z<c then goto not_found;
h:=z-c;@/
@y
@!l,@!r:trie_pointer; {left and right neighbors}
@!ll:1..256; {upper limit of |trie_min| updating}
begin c:=so(trie_c[p]);
z:=trie_min[c]; {get the first conceivably good hole}
loop@+ begin h:=z-c;@/
@z
@x module 956
repeat z:=h+so(trie_c[q]); trie_back(trie_link(z)):=trie_back(z);
trie_link(trie_back(z)):=trie_link(z); trie_link(z):=0; q:=trie_r[q];
@y
repeat z:=h+so(trie_c[q]); l:=trie_back(z); r:=trie_link(z);
trie_back(r):=l; trie_link(l):=r; trie_link(z):=0;
if l<256 then
begin if z<256 then ll:=z @+else ll:=256;
repeat trie_min[l]:=r; incr(l);
until l=ll;
end;
q:=trie_r[q];
@z
@x module 958 is entirely replaced
@y by the following new code:
@ When the whole trie has been allocated into the sequential table, we
must go through it once again so that |trie| contains the correct
information. Null pointers in the linked trie will be represented by the
value~0, which properly implements an ``empty'' family.
@<Move the data into |trie|@>=
h.rh:=0; h.b0:=min_quarterword; h.b1:=min_quarterword; {|trie_link:=0|,
|trie_op:=min_quarterword|, |trie_char:=qi(0)|}
if trie_root=0 then {no patterns were given}
begin for r:=0 to 256 do trie[r]:=h;
trie_max:=256;
end
else begin trie_fix(trie_root); {this fixes the non-holes in |trie|}
r:=0; {now we will zero out all the holes}
repeat s:=trie_link(r); trie[r]:=h; r:=s;
until r>trie_max;
end;
trie_char(0):=qi("?"); {make |trie_char(c)<>c| for all |c|}
@z
@x module 960
@!r,@!s:trie_pointer; {used to clean up the packed |trie|}
@!h:two_halves; {template used to zero out |trie|'s holes}
begin scan_left_brace; {a left brace must follow \.{\\patterns}}
init_pattern_memory;@/
@<Enter all of the patterns into a linked trie, until coming to a right
brace@>;
trie_root:=compress_trie(trie_root); {compress the trie}
@<Pack the trie@>;
end;
@y
begin if trie_not_ready then
begin set_cur_lang; scan_left_brace; {a left brace must follow \.{\\patterns}}
@<Enter all of the patterns into a linked trie, until coming to a right
brace@>;
end
else begin print_err("Too late for "); print_esc("patterns");
help1("All patterns must be given before typesetting begins.");
error; link(garbage):=scan_toks(false,false); flush_list(def_ref);
end;
end;
@z
@x module 963
q:=0;
while l<k do
begin incr(l); c:=hc[l]; p:=trie_l[q]; first_child:=true;
@y
q:=0; hc[0]:=cur_lang;
while l<=k do
begin c:=hc[l]; incr(l); p:=trie_l[q]; first_child:=true;
@z
@x module 966 now says that the first call ``takes'' location 1,
@y and the Pascal code in this module is entirely replaced by the following:
@<Declare procedures for preprocessing hyphenation patterns@>=
procedure init_trie;
var @!p:trie_pointer; {pointer for initialization}
@!j,@!k,@!t:integer; {all-purpose registers for initialization}
@!r,@!s:trie_pointer; {used to clean up the packed |trie|}
@!h:two_halves; {template used to zero out |trie|'s holes}
begin @<Get ready to compress the trie@>;
if trie_root<>0 then
begin first_fit(trie_root); trie_pack(trie_root);
end;
@<Move the data into |trie|@>;
trie_not_ready:=false;
end;
@z
@x module 1033 gets new code before main_loop_1
@y
if mode>0 then if language<>clang then fix_language;
@z
@x modules 1091 and 1200
push_nest; mode:=hmode; space_factor:=1000;
@y
push_nest; mode:=hmode; space_factor:=1000; clang:=0;
@z
@x module 1324
for k:=0 to trie_max do dump_hh(trie[k]);
dump_int(trie_op_ptr);
for k:=min_quarterword+1 to trie_op_ptr do
begin dump_int(hyf_distance[k]);
dump_int(hyf_num[k]);
dump_int(hyf_next[k]);
end;
print_ln; print_int(hyph_count); print(" hyphenation exception");
if hyph_count<>1 then print_char("s");
print_nl("Hyphenation trie of length "); print_int(trie_max);
@.Hyphenation trie...@>
print(" has "); print_int(qo(trie_op_ptr)); print(" op");
if trie_op_ptr<>min_quarterword+1 then print_char("s")
@y
print_ln; print_int(hyph_count); print(" hyphenation exception");
if hyph_count<>1 then print_char("s");
if trie_not_ready then init_trie;
dump_int(trie_max);
for k:=0 to trie_max do dump_hh(trie[k]);
dump_int(trie_op_ptr);
for k:=1 to trie_op_ptr do
begin dump_int(hyf_distance[k]);
dump_int(hyf_num[k]);
dump_int(hyf_next[k]);
end;
print_nl("Hyphenation trie of length "); print_int(trie_max);
@.Hyphenation trie...@>
print(" has "); print_int(trie_op_ptr); print(" op");
if trie_op_ptr<>1 then print_char("s");
print(" out of "); print_int(trie_op_size);
for k:=255 downto 0 do if trie_used[k]>min_quarterword then
begin print_nl(" "); print_int(qo(trie_used[k]));
print(" for language "); print_int(k);
dump_int(k); dump_int(qo(trie_used[k]));
end
@z
@x module 1325
undump_size(0)(trie_size)('trie size')(trie_max);
for k:=0 to trie_max do undump_hh(trie[k]);
undump(min_quarterword)(max_quarterword)(trie_op_ptr);
for k:=min_quarterword+1 to trie_op_ptr do
begin undump(0)(63)(hyf_distance[k]); {a |small_number|}
undump(0)(63)(hyf_num[k]);
undump(min_quarterword)(max_quarterword)(hyf_next[k]);
end
@y
undump_size(0)(trie_size)('trie size')(j); {|trie_max|}
for k:=0 to j do undump_hh(trie[k]);
undump_size(0)(trie_op_size)('trie op size')(j); {|trie_op_ptr|}
for k:=1 to j do
begin undump(0)(63)(hyf_distance[k]); {a |small_number|}
undump(0)(63)(hyf_num[k]);
undump(min_quarterword)(max_quarterword)(hyf_next[k]);
end;
k:=256;
while j>0 do
begin undump(0)(k-1)(k); undump(1)(j)(x); j:=j-x; op_start[k]:=qo(j);
end;
@!init trie_not_ready:=false @+tini
@z
@x module 1341 gets two new definitions
@y
@d language_node=4 {|subtype| in whatsits that change the current language}
@d stored_language(#)==mem[#+1].int {language number, in the range |0..255|}
@z
@x module 1344 gets a new definition and a new Pascal statement
@y
@d set_language_code=5 {command modifier for \.{\\setlanguage}}
primitive("setlanguage",extension,set_language_code);@/
@!@:set_language_}{\.{\\setlanguage} primitive@>
@z
@x module 1346 gets a new case
@y
set_language_code:print_esc("setlanguage");
@z
@x module 1348 gets a new case
@y
set_language_code:@<Implement \.{\\setlanguage}@>;
@z
@x module 1356 gets a new case
@y
language_node:begin print_esc("setlanguage");
print_int(stored_language(p));
end;
@z
% in modules 1357 and 1358, change "close_node" to "close_node,language_node".
@x module 1362 becomes two modules
@ @<Advance \(p)past a whatsit node in the |line_break| loop@>=do_nothing
@y
@ @<Advance \(p)past a whatsit node in the \(l)|line_break| loop@>=
if subtype(cur_p)=language_node then cur_lang:=stored_language(cur_p)
@ @<Advance \(p)past a whatsit node in the \(p)pre-hyphenation loop@>=
if subtype(s)=language_node then cur_lang:=stored_language(s)
@z and old module 1367 is moved to just before the old module 1378
@x module 1373 gets a new case
@y
language_node:do_nothing;
@z
@x new modules before the system-dependent changes (i.e. before the old 1376)
@y
@ @<Implement \.{\\setlanguage}@>=
if abs(mode)<>hmode then report_illegal_case
else begin new_whatsit(language_node,small_node_size);
scan_int;
if cur_val<=0 then clang:=0
else if cur_val>255 then clang:=0
else clang:=cur_val;
stored_language(tail):=clang;
end
@ Finally, we need a subroutine that comes into play when a character of
a non-|clang| language is being appended to the current paragraph.
@<Declare action...@>=
procedure fix_language;
var @!l:ASCII_code; {the new current language}
begin if language<=0 then l:=0
else if language>255 then l:=0
else l:=language;
if l<>clang then
begin new_whatsit(language_node,small_node_size);
stored_language(tail):=l; clang:=l;
end;
end;
@z
362. Major extension to ligature capability.
@x module 143
a linked list of character nodes for those characters.
@y
a linked list of character nodes for all original characters that have been
deleted. (This list might be empty if the characters that generated the
ligature were retained in other nodes.)
The |subtype| field is 0, plus 2 and/or 1 if the original source of the
ligature included implicit left and/or right boundaries.
@z
@x in module 144 add the following procedure to the existing code
@y
function new_lig_item(@!c:quarterword):pointer;
var p:pointer; {the new node}
begin p:=get_node(small_node_size); character(p):=c; lig_ptr(p):=null;
new_lig_item:=p;
end;
@z
@x module 193
font_in_short_display:=font(lig_char(p));
short_display(lig_ptr(p)); print_char(")");
@y
if subtype(p)>1 then print_char("|");
font_in_short_display:=font(lig_char(p)); short_display(lig_ptr(p));
if odd(subtype(p)) then print_char("|");
print_char(")");
end
@z
@x modules 208 and 209
@d radical=65 {square root and similar signs ( \.{\\radical} )}
@y
@d no_boundary=65 {suppress boundary ligatures ( \.{\\noboundary} )}
@d radical=66 {square root and similar signs ( \.{\\radical} )}
@z and so on, adding 1 to each definition until getting to max_command=100
@x module 265 gets a new statement
@y
primitive("noboundary",no_boundary,0);@/
@!@:no_boundary_}{\.{\\noboundary} primitive@>
@z
@x module 266 gets a new case
@y
no_boundary:print_esc("noboundary");
@z
@x module 545 is entirely replaced
@y by the following new specifications:
@ The |lig_kern| array contains instructions in a simple programming language
that explains what to do for special letter pairs. Each word in this array is a
|@!lig_kern_command| of four bytes.
\yskip\hang first byte: |skip_byte|, indicates that this is the final program
step if the byte is 128 or more, otherwise the next step is obtained by
skipping this number of intervening steps.\par
\hang second byte: |next_char|, ``if |next_char| follows the current character,
then perform the operation and stop, otherwise continue.''\par
\hang third byte: |op_byte|, indicates a ligature step if less than~128,
a kern step otherwise.\par
\hang fourth byte: |remainder|.\par
\yskip\noindent
In a kern step, an
additional space equal to |kern[256*(op_byte-128)+remainder]| is inserted
between the current character and |next_char|. This amount is
often negative, so that the characters are brought closer together
by kerning; but it might be positive.
There are eight kinds of ligature steps, having |op_byte| codes $4a+2b+c$ where
$0\le a\le b+c$ and $0\le b,c\le1$. The character whose code is
|remainder| is inserted between the current character and |next_char|;
then the current character is deleted if $b=0$, and |next_char| is
deleted if $c=0$; then we pass over $a$~characters to reach the next
current character (which may have a ligature/kerning program of its own).
If the very first instruction of the |lig_kern| array has |skip_byte=255|,
the |next_char| byte is the so-called right boundary character of this font;
the value of |next_char| need not lie between |bc| and~|ec|.
If the very last instruction of the |lig_kern| array has |skip_byte=255|,
there is a special ligature/kerning program for a left boundary character,
beginning at location |256*op_byte+remainder|.
The interpretation is that \TeX\ puts implicit boundary characters
before and after each consecutive string of characters from the same font.
These implicit characters do not appear in the output, but they can affect
ligatures and kerning.
If the very first instruction of a character's |lig_kern| program has
|skip_byte>128|, the program actually begins in location
|256*op_byte+remainder|. This feature allows access to large |lig_kern|
arrays, because the first instruction must otherwise
appear in a location |<=255|.
Any instruction with |skip_byte>128| in the |lig_kern| array must have
|256*op_byte+remainder<nl|. If such an instruction is encountered during
normal program execution, it denotes an unconditional halt; no ligature
command is performed.
@d stop_flag=qi(128) {value indicating `\.{STOP}' in a lig/kern program}
@d kern_flag=qi(128) {op code for a kern step}
@d skip_byte(#)==#.b0
@d next_char(#)==#.b1
@d op_byte(#)==#.b2
@d rem_byte(#)==#.b3
@z
@x module 549
@<Glob...@>=
@y
@d non_char==qi(256) {a |halfword| code that can't match a real character}
@d non_address==font_mem_size {a spurious |font_index|}
@<Glob...@>=
@z
@x module 548 gets a new type definition
@y
@!font_index=0..font_mem_size;
@z
@x module 549
{current \.{\\skewchar} values}
@y
{current \.{\\skewchar} values}
@!bchar_label:array[internal_font_number] of font_index;
{start of |lig_kern| program for left boundary character,
|non_address| if there is none}
@!font_bchar:array[internal_font_number] of min_quarterword..non_char;
{right boundary character, |non_char| if there is none}
@!font_false_bchar:array[internal_font_number] of min_quarterword..non_char;
{|font_bchar| if it doesn't exist in the font, otherwise |non_char|}
@z
@x module 557
@d char_kern_end(#)==rem_byte(#)].sc
@y NOTE: Optimize kern_base_offset in your change file! It's a constant.
@d char_kern_end(#)==256*op_byte(#)+rem_byte(#)].sc
@d kern_base_offset==256*(kern_flag)
@d lig_kern_restart_end(#)==256*op_byte(#)+rem_byte(#)+32768-kern_base_offset
@d lig_kern_restart(#)==lig_kern_base[#]+lig_kern_restart_end
@z
@x module 560
@!z:scaled; {the design size or the ``at'' size}
@y
@!bch_label:integer; {left boundary start location, or infinity}
@!bchar:0..256; {right boundary character, or 256}
@!z:scaled; {the design size or the ``at'' size}
@z
@x module 566
kern_base[f]:=lig_kern_base[f]+nl;
exten_base[f]:=kern_base[f]+nk;
@y
kern_base[f]:=lig_kern_base[f]+nl-kern_base_offset;
exten_base[f]:=kern_base[f]+kern_base_offset+nk;
@z
@x module 573: The entire module is replaced
@y by the following code.
@ @d check_existence(#)==@t@>@;@/
begin check_byte_range(#);
qw:=char_info(f)(#); {N.B.: not |qi(#)|}
if not char_exists(qw) then abort;
end
@<Read ligature/kern program@>=
bch_label:=@'77777; bchar:=256;
if nl>0 then
begin for k:=lig_kern_base[f] to kern_base[f]+kern_base_offset-1 do
begin store_four_quarters(font_info[k].qqqq);
if a>128 then
begin if 256*c+d>=nl then abort;
if a=255 then if k=lig_kern_base[f] then bchar:=b;
end
else begin if b<>bchar then check_existence(b);
if c<128 then check_existence(d) {check ligature}
else if 256*(c-128)+d>=nk then abort; {check kern}
if a<128 then if k-lig_kern_base[f]+a+1>=nl then abort;
end;
end;
if a=255 then bch_label:=256*c+d;
end;
for k:=kern_base[f]+kern_base_offset to exten_base[f]-1 do
store_scaled(font_info[k].sc);
@z
@x module 574 (actually a bugfix)
if a<>0 then check_byte_range(a);
if b<>0 then check_byte_range(b);
if c<>0 then check_byte_range(c);
check_byte_range(d);
@y
if a<>0 then check_existence(a);
if b<>0 then check_existence(b);
if c<>0 then check_existence(c);
check_existence(d);
@z
@x module 576
font_name[f]:=nom;
@y
if bch_label<nl then bchar_label[f]:=bch_label+lig_kern_base[f]
else bchar_label[f]:=non_address;
font_bchar[f]:=qi(bchar);
font_false_bchar[f]:=qi(bchar);
if bchar<=ec then if bchar>=bc then
begin qw:=char_info(f)(bchar); {N.B.: not |qi(bchar)|}
if char_exists(qw) then font_false_bchar[f]:=non_char;
end;
font_name[f]:=nom;
@z
@x module 708 (slight but optional optimization)
continue: if (qo(y)>=font_bc[g])and(qo(y)<=font_ec[g]) then
begin q:=char_info(g)(y);
@y
if (qo(y)>=font_bc[g])and(qo(y)<=font_ec[g]) then
begin continue: q:=char_info(g)(y);
@z
@x module 740 (another bugfix)
i:=char_info(f)(y);
@y
i:=char_info(f)(y);
if not char_exists(i) then goto done;
@z
@x module 741
repeat cur_i:=font_info[a].qqqq;
if qo(next_char(cur_i))=skew_char[cur_f] then
begin if op_bit(cur_i)>=kern_flag then
s:=char_kern(cur_f)(cur_i);
goto done1;
end;
incr(a);
until stop_bit(cur_i)>=stop_flag;
@y
cur_i:=font_info[a].qqqq;
if skip_byte(cur_i)>stop_flag then
begin a:=256*(qo(op_byte(cur_i)))+rem_byte(cur_i);
cur_i:=font_info[a].qqqq;
end;
loop begin if qo(next_char(cur_i))=skew_char[cur_f] then
begin if op_byte(cur_i)>=kern_flag then
if skip_byte(cur_i)<=stop_flag then s:=char_kern(cur_f)(cur_i);
goto done1;
end;
if skip_byte(cur_i)>=stop_flag then goto done1
else a:=a+qo(skip_byte(cur_i))+1;
cur_i:=font_info[a].qqqq;
end;
@z
@x module 749 (yet another bugfix of the same type!)
@!p,@!v,@!x,@!y,@!z:pointer; {temporary registers for box construction}
@y
@!p,@!v,@!x,@!y,@!z:pointer; {temporary registers for box construction}
@!c:quarterword;@+@!i:four_quarters; {registers for character examination}
@z
@x ibid
begin cur_c:=rem_byte(cur_i); character(nucleus(q)):=cur_c;
cur_i:=char_info(cur_f)(cur_c);
@y
begin c:=rem_byte(cur_i); i:=char_info(cur_f)(c);
if char_exists(i) then
begin cur_c:=c; cur_i:=i; character(nucleus(q)):=c;
end;
@z
@x module 752
@!p:pointer; {temporary register for list manipulation}
@y
@!p,@!r:pointer; {temporary registers for list manipulation}
@z
@x ibid
repeat cur_i:=font_info[a].qqqq;@/
@<If instruction |cur_i| is a kern with |cur_c|,
attach the kern after |q| and |return|;
or if it is a ligature with |cur_c|, combine
noads |q| and |p| and |goto restart|@>;
incr(a);
until stop_bit(cur_i)>=stop_flag;
@y
cur_i:=font_info[a].qqqq;
if skip_byte(cur_i)>stop_flag then
begin a:=lig_kern_restart(cur_f)(cur_i);
cur_i:=font_info[a].qqqq;
end;
loop@+ begin @<If instruction |cur_i| is a kern with |cur_c|, attach
the kern after~|q|; or if it is a ligature with |cur_c|, combine
noads |q| and~|p| appropriately; then |return| if the cursor has
moved past a noad, or |goto restart|@>;
if skip_byte(cur_i)>=stop_flag then return;
a:=a+qo(skip_byte(cur_i))+1;
cur_i:=font_info[a].qqqq;
end;
@z
@x The entire code of module 753 is revised
@y and should be replaced by the following:
@ Note that a ligature between an |ord_noad| and another kind of noad
is replaced by an |ord_noad|, when the two noads collapse into one.
But we could make a parenthesis (say) change shape when it follows
certain letters. Presumably a font designer will define such
ligatures only when this convention makes sense.
\chardef\?='174 % vertical line to indicate character retention
@<If instruction |cur_i| is a kern with |cur_c|, ...@>=
if next_char(cur_i)=cur_c then if skip_byte(cur_i)<=stop_flag then
if op_byte(cur_i)>=kern_flag then
begin p:=new_kern(char_kern(cur_f)(cur_i));
link(p):=link(q); link(q):=p; return;
end
else begin check_interrupt; {allow a way out of infinite ligature loop}
case op_byte(cur_i) of
qi(1),qi(5): character(nucleus(q)):=rem_byte(cur_i); {\.{=:\?}, \.{=:\?>}}
qi(2),qi(6): character(nucleus(p)):=rem_byte(cur_i); {\.{\?=:}, \.{\?=:>}}
qi(3),qi(7),qi(11):begin r:=new_noad; {\.{\?=:\?}, \.{\?=:\?>}, \.{\?=:\?>>}}
character(nucleus(r)):=rem_byte(cur_i);
fam(nucleus(r)):=fam(nucleus(q));@/
link(q):=r; link(r):=p;
if op_byte(cur_i)<qi(11) then math_type(nucleus(r)):=math_char
else math_type(nucleus(r)):=math_text_char; {prevent combination}
end;
othercases begin link(q):=link(p);
character(nucleus(q)):=rem_byte(cur_i); {\.{=:}}
mem[subscr(q)]:=mem[subscr(p)]; mem[supscr(q)]:=mem[supscr(p)];@/
free_node(p,noad_size);
end
endcases;
if op_byte(cur_i)>qi(3) then return;
math_type(nucleus(q)):=math_char; goto restart;
end
@z
@x module 862
@!q,@!r,@!s:pointer; {miscellaneous nodes of temporary interest}
@y
@!q,@!r,@!s,@!prev_s:pointer; {miscellaneous nodes of temporary interest}
@z
@x module 892
nodes $p_a$ and~$p_b$ in the description above are placed into variables
@y
nodes $p_{a-1}$ and~$p_b$ in the description above are placed into variables
@z
@x ibid
@!hc:array[0..65] of halfword; {word to be hyphenated}
@y
@!hc:array[0..65] of 0..256; {word to be hyphenated}
@z
@x ibid
@!hu:array[1..63] of ASCII_code; {like |hc|, before conversion to lowercase}
@y
@!hu:array[0..63] of 0..256; {like |hc|, before conversion to lowercase}
@z
@x module 894
s:=link(cur_p);
@y
prev_s:=cur_p; s:=link(prev_s);
@z
@x module 895
label done,found,not_found,found1,exit;
@y
label common_ending,done,found,found1,not_found,not_found+1,exit;
@z
@x module 896
else if type(s)=ligature_node then
begin q:=lig_ptr(s); c:=qo(character(q)); hf:=font(q);
end
@y
else if type(s)=ligature_node then
if lig_ptr(s)=null then goto continue
else begin q:=lig_ptr(s); c:=qo(character(q)); hf:=font(q);
end
@z
@x ibid
continue: s:=link(s);
@y
continue: prev_s:=s; s:=link(prev_s);
@z
@x ibid
ha:=s
@y
ha:=prev_s
@z
@x module 898
begin j:=hn; q:=lig_ptr(s);
if font(q)<>hf then goto done3;
repeat c:=qo(character(q));
if lc_code(c)=0 then goto done3;
if j=63 then goto done3;
incr(j); hu[j]:=c; hc[j]:=lc_code(c);@/
q:=link(q);
until q=null;
@y
begin if font(lig_char(s))<>hf then goto done3;
j:=hn; q:=lig_ptr(s);
while q>null do
begin c:=qo(character(q));
if lc_code(c)=0 then goto done3;
if j=63 then goto done3;
incr(j); hu[j]:=c; hc[j]:=lc_code(c);@/
q:=link(q);
end;
@z
@x module 900 gets three new global variables
@y
@!init_list:pointer; {list of punctuation characters preceding the word}
@!init_lig:boolean; {does |init_list| represent a ligature?}
@!init_lft:boolean; {if so, did the ligature involve a left boundary?}
@z
@x module 901
@!q,@!r,@!s:pointer; {temporary registers for list manipulation}
@y
@!p,@!q,@!r,@!s:pointer; {temporary registers for list manipulation}
@!bchar:halfword; {right boundary character of hyphenated word, or |non_char|}
@z
@x module 903 should be entirely replaced
@y by the following:
@ If hyphens are in fact going to be inserted, \TeX\ first deletes the
subsequence of nodes between |ha| and~|hb|. An attempt is made to
preserve the effect that implicit boundary characters and punctuation marks
had on ligatures inside the hyphenated word, by storing a left boundary or
preceding character in |hu[0]| and by storing a possible right boundary
in |bchar|. We set |j:=0| if |hu[0]| is to be part of the reconstruction;
otherwise |j:=1|.
The variable |s| will point to the tail of the current hlist, and
|q| will point to the node following |hb|, so that
things can be hooked up after we reconstitute the hyphenated word.
@<Replace nodes |ha..hb| by a sequence of nodes...@>=
q:=link(hb); link(hb):=null; r:=link(ha); link(ha):=null; bchar:=non_char;
if type(hb)=ligature_node then if odd(subtype(hb)) then
bchar:=font_bchar[hf];
if is_char_node(ha) then
begin init_list:=ha; init_lig:=false; hu[0]:=qo(character(ha));
end
else if type(ha)=ligature_node then
begin init_list:=lig_ptr(ha); init_lig:=true; init_lft:=(subtype(ha)>1);
hu[0]:=qo(character(lig_char(ha)));
if init_list=null then if init_lft then
begin hu[0]:=256; init_lig:=false;
end; {in this case a ligature will be reconstructed from scratch}
free_node(ha,small_node_size);
end
else goto not_found+1; {no punctuation found}
s:=cur_p; {we have |cur_p<>ha| because |type(cur_p)=glue_node|}
while link(s)<>ha do s:=link(s);
j:=0; goto common_ending;
not_found+1: j:=1; s:=ha; init_list:=null;
if not is_char_node(r) then if type(r)=ligature_node then
if subtype(r)>1 then
begin j:=0; hu[0]:=256; init_lig:=false;
end;
common_ending: flush_node_list(r);
@<Reconstitute nodes for the hyphenated word, inserting discretionary hyphens@>;
flush_list(init_list)
@z
@x modules 905--911 are entirely replaced
@y by the following new code:
Still further complications arise in the presence of ligatures that do not
delete the original characters. When punctuation precedes the word being
hyphenated, \TeX's method is not perfect under all possible scenarios,
because punctuation marks and letters can propagate information back and forth.
For example, suppose the original pre-hyphenation pair
\.{*a} changes to \.{*y} via a \.{\?=:} ligature, which changes to \.{xy}
via a \.{=:\?} ligature; if $p_{a-1}=\.x$ and $p_a=\.y$, the reconstitution
procedure isn't smart enough to obtain \.{xy} again. In such cases the
font designer should include a ligature that goes from \.{xa} to \.{xy}.
@ The processing is facilitated by a subroutine called |reconstitute|. Given
a string of characters $x_j\ldots x_n$, there is a smallest index $m\ge j$
such that the ``translation'' of $x_j\ldots x_n$ by ligatures and kerning
has the form $y_1\ldots y_t$ followed by the translation of $x_{m+1}\ldots x_n$,
where $y_1\ldots y_t$ is some nonempty sequence of character, ligature, and
kern nodes. We call $x_j\ldots x_m$ a ``cut prefix'' of $x_j\ldots x_n$.
For example, if $x_1x_2x_3=\.{fly}$, and if the font contains `fl' as a
ligature and a kern between `fl' and `y', then $m=2$, $y=2$, and $y_1$ will
be a ligature node for `fl' followed by an appropriate kern node~$y_2$.
In the most common case, $x_j$~forms no ligature with $x_{j+1}$ and we
simply have $m=j$, $y_1=x_j$. If $m<n$ we can repeat the procedure on
$x_{m+1}\ldots x_n$ until the entire translation has been found.
The |reconstitute| function returns the integer $m$ and puts the nodes
$y_1\ldots y_t$ into a linked list starting at |link(hold_head)|,
getting the input $x_j\ldots x_n$ from the |hu| array. If $x_j=256$,
we consider $x_j$ to be an implicit left boundary character; in this
case |j| must be strictly less than~|n|. There is a
parameter |bchar|, which is either 256 or an implicit right boundary character
assumed to be present just following~$x_n$. (The value |hu[n+1]| is never
explicitly examined, but the algorithm imagines that |bchar| is there.)
If there exists an index |k| in the range $j\le k\le m$ such that |hyf[k]|
is odd and such that the result of reconstitute would have been different
if $x_{k+1}$ had been |hchar|, then |reconstitute| sets |hyphen_passed|
to the smallest such~|k|. Otherwise it sets |hyphen_passed| to zero.
A special convention is used in the case |j=0|: Then we assume that the
translation of |hu[0]| appears in a special list of charnodes starting at
|init_list|; moreover, if |init_lig| is |true|, then |hu[0]| will be
a ligature character, involving a left boundary if |init_lft| is |true|.
This facility is provided for cases when a hyphenated
word is preceded by punctuation (like single or double quotes) that might
affect the translation of the beginning of the word.
@<Glob...@>=
@!hyphen_passed:small_number; {first hyphen in a ligature, if any}
@ @<Declare the function called |reconstitute|@>=
function reconstitute(@!j,@!n:small_number;@!bchar,@!hchar:halfword):
small_number;
label continue,done;
var @!p:pointer; {temporary register for list manipulation}
@!t:pointer; {a node being appended to}
@!q:four_quarters; {character information or a lig/kern instruction}
@!cur_rh:halfword; {hyphen character for ligature testing}
@!test_char:halfword; {hyphen or other character for ligature testing}
@!w:scaled; {amount of kerning}
@!k:font_index; {position of current lig/kern instruction}
begin hyphen_passed:=0; t:=hold_head; w:=0; link(hold_head):=null;
{at this point |ligature_present=lft_hit=rt_hit=false|}
@<Set up data structures with the cursor following position |j|@>;
continue:@<If there's a ligature or kern at the cursor position, update the data
structures, possibly advancing~|j|; continue until the cursor moves@>;
@<Append a ligature and/or kern to the translation;
|goto continue| if the stack of inserted ligatures is nonempty@>;
reconstitute:=j;
end;
@ The reconstitution procedure shares many of the global data structures
by which \TeX\ has processed the words before they were hyphenated.
There is an implied ``cursor'' between characters |cur_l| and |cur_r|;
these characters will be tested for possible ligature activity. If
|ligature_present| then |cur_l| is a ligature character formed from the
original characters following |cur_q| in the current translation list.
There is a ``ligature stack'' between the cursor and character |j+1|,
consisting of pseudo-ligature nodes linked together by their |link| fields.
This stack is normally empty unless a ligature command has created a new
character that will need to be processed later. A pseudo-ligature is
a special node having a |character| field that represents a potential
ligature and a |lig_ptr| field that points to a |char_node| or is |null|.
We have
$$|cur_r|=\cases{|character(lig_stack)|,&if |lig_stack>null|;\cr
|qi(hu[j+1])|,&if |lig_stack=null| and |j<n|;\cr
bchar,&if |lig_stack=null| and |j=n|.\cr}$$
@<Glob...@>=
@!cur_l,@!cur_r:halfword; {characters before and after the cursor}
@!cur_q:pointer; {where a ligature should be detached}
@!lig_stack:pointer; {unfinished business to the right of the cursor}
@!ligature_present:boolean; {should a ligature node be made for |cur_l|?}
@!lft_hit,@!rt_hit:boolean; {did we hit a ligature with a boundary character?}
@ @d append_charnode_to_t(#)== begin link(t):=get_avail; t:=link(t);
font(t):=hf; character(t):=#;
end
@d set_cur_r==begin if j<n then cur_r:=qi(hu[j+1])@+else cur_r:=bchar;
if odd(hyf[j]) then cur_rh:=hchar@+else cur_rh:=non_char;
end
@<Set up data structures with the cursor following position |j|@>=
cur_l:=qi(hu[j]); cur_q:=t;
if j=0 then
begin ligature_present:=init_lig; p:=init_list;
if ligature_present then lft_hit:=init_lft;
while p>null do
begin append_charnode_to_t(character(p)); p:=link(p);
end;
end
else if cur_l<non_char then append_charnode_to_t(cur_l);
lig_stack:=null; set_cur_r
@ We may want to look at the lig/kern program twice, once for a hyphen
and once for a normal letter. (The hyphen might appear after the letter
in the program, so we'd better not try to look for both at once.)
@<If there's a ligature or kern at the cursor position, update...@>=
if cur_l=non_char then
begin k:=bchar_label[hf];
if k=non_address then goto done@+else q:=font_info[k].qqqq;
end
else begin q:=char_info(hf)(cur_l);
if char_tag(q)<>lig_tag then goto done;
k:=lig_kern_start(hf)(q); q:=font_info[k].qqqq;
if skip_byte(q)>stop_flag then
begin k:=lig_kern_restart(hf)(q); q:=font_info[k].qqqq;
end;
end; {now |k| is the starting address of the lig/kern program}
if cur_rh<non_char then test_char:=cur_rh@+else test_char:=cur_r;
loop@+begin if next_char(q)=test_char then if skip_byte(q)<=stop_flag then
if cur_rh<non_char then
begin hyphen_passed:=j; hchar:=non_char; cur_rh:=non_char;
goto continue;
end
else begin if hchar<non_char then if odd(hyf[j]) then
begin hyphen_passed:=j; hchar:=non_char;
end;
if op_byte(q)<kern_flag then
@<Carry out a ligature replacement, updating the cursor structure
and possibly advancing~|j|; |goto continue| if the cursor doesn't
advance, otherwise |goto done|@>;
w:=char_kern(hf)(q); goto done; {this kern will be inserted below}
end;
if skip_byte(q)>=stop_flag then
if cur_rh=non_char then goto done
else begin cur_rh:=non_char; goto continue;
end;
k:=k+qo(skip_byte(q))+1; q:=font_info[k].qqqq;
end;
done:
@ @d wrap_lig(#)==if ligature_present then
begin p:=new_ligature(hf,cur_l,link(cur_q));
if lft_hit then
begin subtype(p):=2; lft_hit:=false;
end;
if # then if lig_stack=null then
begin incr(subtype(p)); rt_hit:=false;
end;
link(cur_q):=p; t:=p; ligature_present:=false;
end
@d pop_lig_stack==begin if lig_ptr(lig_stack)>null then
begin link(t):=lig_ptr(lig_stack); {this is a charnode for |hu[j+1]|}
t:=link(t); incr(j);
end;
p:=lig_stack; lig_stack:=link(p); free_node(p,small_node_size);
if lig_stack=null then set_cur_r@+else cur_r:=character(lig_stack);
end {if |lig_stack| isn't |null| we have |cur_rh=non_char|}
@<Append a ligature and/or kern to the translation...@>=
wrap_lig(rt_hit);
if w<>0 then
begin link(t):=new_kern(w); t:=link(t); w:=0;
end;
if lig_stack>null then
begin cur_q:=t; cur_l:=character(lig_stack); ligature_present:=true;
pop_lig_stack; goto continue;
end
@ @<Carry out a ligature replacement, updating the cursor structure...@>=
begin if cur_l=non_char then lft_hit:=true;
if j=n then if lig_stack=null then rt_hit:=true;
check_interrupt; {allow a way out in case there's an infinite ligature loop}
case op_byte(q) of
qi(1),qi(5):begin cur_l:=rem_byte(q); {\.{=:\?}, \.{=:\?>}}
ligature_present:=true;
end;
qi(2),qi(6):begin cur_r:=rem_byte(q); {\.{\?=:}. \.{\?=:>}}
if lig_stack>null then character(lig_stack):=cur_r
else begin lig_stack:=new_lig_item(cur_r);
if j=n then bchar:=non_char
else begin p:=get_avail; lig_ptr(lig_stack):=p;
character(p):=qi(hu[j+1]); font(p):=hf;
end;
end;
end;
qi(3):begin cur_r:=rem_byte(q); {\.{\?=:\?}}
p:=lig_stack; lig_stack:=new_lig_item(cur_r); link(lig_stack):=p;
end;
qi(7),qi(11):begin wrap_lig(false); {\.{\?=:\?>}, \.{\?=:\?>>}}
cur_q:=t; cur_l:=rem_byte(q); ligature_present:=true;
end;
othercases begin cur_l:=rem_byte(q); ligature_present:=true; {\.{=:}}
if lig_stack>null then pop_lig_stack
else if j=n then goto done
else begin append_charnode_to_t(cur_r); incr(j); set_cur_r;
end;
end
endcases;
if op_byte(q)>qi(4) then if op_byte(q)<>qi(7) then goto done;
goto continue;
end
@z
@x module 912
@!c:ASCII_code; {character temporarily replaced by a hyphen}
@y
@!c:ASCII_code; {character temporarily replaced by a hyphen}
@!c_loc:0..63; {where that character came from}
@!r_count:integer; {replacement count for discretionary}
@z
@x modules 913--918 are to be completely replaced
@y by the following code:
@ When the following code is performed, |hyf[0]| and |hyf[hn]| will be zero.
@<Reconstitute nodes for the hyphenated word...@>=
repeat l:=j; j:=reconstitute(j,hn,bchar,qi(hyf_char))+1;
if hyphen_passed=0 then
begin link(s):=link(hold_head);
while link(s)>null do s:=link(s);
if odd(hyf[j-1]) then
begin l:=j; hyphen_passed:=j-1; link(hold_head):=null;
end;
end;
if hyphen_passed>0 then
@<Create and append a discretionary node as an alternative to the
unhyphenated word, and continue to develop both branches until they
become equivalent@>;
until j>hn;
link(s):=q
@ @d advance_major_tail==begin major_tail:=link(major_tail); incr(r_count);
end
@<Create and append a discretionary node as an alternative...@>=
begin r:=get_node(small_node_size);
link(r):=link(hold_head); type(r):=disc_node;
major_tail:=r; r_count:=0;
while link(major_tail)>null do advance_major_tail;
i:=hyphen_passed;
@<Put the \(c)characters |hu[l..i]| and a hyphen into |pre_break(r)|@>;
@<Put the \(c)characters |hu[i+1..@,]| into |post_break(r)|, appending to this
list and to |major_tail| until synchronization has been achieved@>;
@<Move pointer |s| to the end of the current list, and set |replace_count(r)|
appropriately@>;
end
@ The new hyphen might combine with the previous character via ligature
or kern. At this point we have |l-1<=i<=j| and |i<hn|.
@<Put the \(c)characters |hu[l..i]| and a hyphen into |pre_break(r)|@>=
minor_tail:=null; pre_break(r):=null; hyf_node:=new_character(hf,hyf_char);
if hyf_node<>null then
begin incr(i); c:=hu[i]; hu[i]:=hyf_char; free_avail(hyf_node);
end;
while l<=i do
begin l:=reconstitute(l,i,font_bchar[hf],non_char)+1;
if link(hold_head)>null then
begin if minor_tail=null then pre_break(r):=link(hold_head)
else link(minor_tail):=link(hold_head);
minor_tail:=link(hold_head);
while link(minor_tail)>null do minor_tail:=link(minor_tail);
end;
end;
if hyf_node<>null then
begin hu[i]:=c; {restore the character in the hyphen position}
l:=i; decr(i);
end
@ The synchronization algorithm begins with |l=i+1<=j|.
@<Put the \(c)characters |hu[i+1..@,]| into |post_break(r)|...@>=
minor_tail:=null; post_break(r):=null; c_loc:=0;
if bchar_label[hf]<non_address then {put left boundary at beginning of new line}
begin decr(l); c:=hu[l]; c_loc:=l; hu[l]:=256;
end;
while l<j do
begin repeat l:=reconstitute(l,hn,bchar,non_char)+1;
if c_loc>0 then
begin hu[c_loc]:=c; c_loc:=0;
end;
if link(hold_head)>null then
begin if minor_tail=null then post_break(r):=link(hold_head)
else link(minor_tail):=link(hold_head);
minor_tail:=link(hold_head);
while link(minor_tail)>null do minor_tail:=link(minor_tail);
end;
until l>=j;
while l>j do
@<Append characters of |hu[j..@,]| to |major_tail|, advancing~|j|@>;
end
@ @<Append characters of |hu[j..@,]|...@>=
begin j:=reconstitute(j,hn,bchar,non_char)+1;
link(major_tail):=link(hold_head);
while link(major_tail)>null do advance_major_tail;
end
@ Ligature insertion can cause a word to grow exponentially in size. Therefore
we must test the size of |r_count| here, even though the hyphenated text
was at most 63 characters long.
@<Move pointer |s| to the end of the current list...@>=
if r_count>127 then {we have to forget the discretionary hyphen}
begin link(s):=link(r); link(r):=null; flush_node_list(r);
end
else begin link(s):=r; replace_count(r):=r_count;
end;
s:=major_tail
@z
@x module 1030
@d main_loop=70 {go here to typeset |cur_chr| in the current font}
@d main_loop_1=71 {like |main_loop|, but |(f,c)| = current font and char}
@d main_loop_2=72 {like |main_loop_1|, but |c| is known to be in range}
@d main_loop_3=73 {like |main_loop_2|, but several variables are set up}
@d append_normal_space=74 {go here to append a normal space between words}
@y
@d main_loop=70 {go here to typeset a string of consecutive characters}
@d main_loop_wrapup=80 {go here to finish a character or ligature}
@d main_loop_move=90 {go here to advance the ligature cursor}
@d main_loop_move_lig=95 {same, when advancing past a generated ligature}
@d main_loop_lookahead=100 {go here to bring in another character, if any}
@d main_lig_loop=110 {go here to check for ligatures or kerning}
@d append_normal_space=120 {go here to append a normal space between words}
@z
@x ibid
label big_switch,reswitch,main_loop,main_loop_1,main_loop_2,main_loop_3,
@y
label big_switch,reswitch,main_loop,main_loop_wrapup,
main_loop_move,main_loop_move+1,main_loop_move+2,main_loop_move_lig,
main_loop_lookahead,main_loop_lookahead+1,
main_lig_loop,main_lig_loop+1,main_lig_loop+2,
@z
@x ibid
@<Local variables for the inner loop of |main_control|@>@;
@y
@z
@x ibid
hmode+char_num: begin scan_char_num; cur_chr:=cur_val; goto main_loop;
end;
@y
hmode+char_num: begin scan_char_num; cur_chr:=cur_val; goto main_loop;@+end;
hmode+no_boundary: begin get_x_token;
if (cur_cmd=letter)or(cur_cmd=other_char)or(cur_cmd=char_given)or
(cur_cmd=char_num) then cancel_boundary:=true;
goto reswitch;
end;
@z
@x modules 1032--1040 are all to be replaced
@y by the following code:
@ The following part of the program was first written in a structured
manner, according to the philosophy that ``premature optimization is
the root of all evil.'' Then it was rearranged into pieces of
spaghetti so that the most common actions could proceed with little or
no redundancy.
The original unoptimized form of this algorithm resembles the
|reconstitute| procedure, which was described earlier in connection with
hyphenation. Again we have an implied ``cursor`` between characters
|cur_l| and |cur_r|. The main difference is that the |lig_stack| can now
contain a charnode as well as pseudo-ligatures; that stack is now
usually nonempty, because the next character of input (if any) has been
appended to it. In |main_control| we have
$$|cur_r|=\cases{|character(lig_stack)|,&if |lig_stack>null|;\cr
|font_bchar[cur_font]|,&otherwise.\cr}$$
Several additional global variables are needed.
@<Glob...@>=
@!main_f:internal_font_number; {the current font}
@!main_i:four_quarters; {character information bytes for |cur_l|}
@!main_j:four_quarters; {ligature/kern command}
@!main_k:font_index; {index into |font_info|}
@!main_p:pointer; {temporary register for list manipulation}
@!main_s:integer; {space factor value}
@!bchar:halfword; {right boundary character of current font, or |non_char|}
@!false_bchar:halfword; {nonexistent character matching |bchar|, or |non_char|}
@!cancel_boundary:boolean; {should the left boundary be ignored?}
@!ins_disc:boolean; {should we insert a discretionary node?}
@ The boolean variables of the main loop are normally false, and always reset
to false before the loop is left. That saves us the extra work of initializing
each time.
@<Set init...@>=
ligature_present:=false; cancel_boundary:=false; lft_hit:=false; rt_hit:=false;
ins_disc:=false;
@ We leave |space_factor| unchanged if |sf_code(cur_chr)=0|; otherwise we
set it to |sf_code(cur_chr)|, except that the space factor never changes
from a value less than 1000 to a value exceeding 1000. The most common
case is |sf_code(cur_chr)=1000|, so we want that case to be fast.
The overall structure of the main loop is presented here. Some program labels
are inside the individual sections.
@d adjust_space_factor==@t@>@;@/
main_s:=sf_code(cur_chr);
if main_s=1000 then space_factor:=1000
else if main_s<1000 then
begin if main_s>0 then space_factor:=main_s;
end
else if space_factor<1000 then space_factor:=1000
else space_factor:=main_s
@<Append character |cur_chr|...@>=
adjust_space_factor;@/
main_f:=cur_font;
bchar:=font_bchar[main_f]; false_bchar:=font_false_bchar[main_f];
if mode>0 then if language<>clang then fix_language;
fast_get_avail(lig_stack); font(lig_stack):=main_f; cur_l:=qi(cur_chr);
character(lig_stack):=cur_l;@/
cur_q:=tail;
if cancel_boundary then
begin cancel_boundary:=false; main_k:=non_address;
end
else main_k:=bchar_label[main_f];
if main_k=non_address then goto main_loop_move+2; {no left boundary processing}
cur_r:=cur_l; cur_l:=non_char;
goto main_lig_loop+1; {begin with cursor after left boundary}
@#
main_loop_wrapup:@<Make a ligature node, if |ligature_present|;
insert a null discretionary, if appropriate@>;
main_loop_move:@<If the cursor is immediately followed by the right boundary,
|goto reswitch|; if it's followed by an invalid character, |goto big_switch|;
otherwise move the cursor one step to the right and |goto main_lig_loop|@>;
main_loop_lookahead:@<Look ahead for another character, or leave |lig_stack|
empty if there's none there@>;
main_lig_loop:@<If there's a ligature/kern command relevant to |cur_l| and
|cur_r|, adjust the text appropriately; exit to |main_loop_wrapup|@>;
main_loop_move_lig:@<Move the cursor past a pseudo-ligature, then
|goto main_loop_lookahead| or |main_lig_loop|@>
@ If the current horizontal list is empty, the reference to |character(tail)|
here is not strictly legal, since |tail| will be a node freshly returned by
|get_avail|. But this should cause no problem on most implementations, and we
do want the inner loop to be fast.
@^dirty Pascal@>
A discretionary break is not inserted for an explicit hyphen when we are in
restricted horizontal mode. In particular, this avoids putting discretionary
nodes inside of other discretionaries.
@d pack_lig(#)== {the parameter is either |rt_hit| or |false|}
begin main_p:=new_ligature(main_f,cur_l,link(cur_q));
if lft_hit then
begin subtype(main_p):=2; lft_hit:=false;
end;
if # then if lig_stack=null then
begin incr(subtype(main_p)); rt_hit:=false;
end;
link(cur_q):=main_p; tail:=main_p; ligature_present:=false;
end
@d wrapup(#)==if cur_l<non_char then
begin if character(tail)=qi(hyphen_char[main_f]) then if link(cur_q)>null then
ins_disc:=true;
if ligature_present then pack_lig(#);
if ins_disc then
begin ins_disc:=false;
if mode>0 then tail_append(new_disc);
end;
end
@<Make a ligature node, if |ligature_present|;...@>=
wrapup(rt_hit)
@ @<If the cursor is immediately followed by the right boundary...@>=
if lig_stack=null then goto reswitch;
cur_q:=tail; cur_l:=cur_r; {or |character(lig_stack)|}
main_loop_move+1:if not is_char_node(lig_stack) then goto main_loop_move_lig;
main_loop_move+2:if(cur_chr<font_bc[main_f])or(cur_chr>font_ec[main_f]) then
begin char_warning(main_f,cur_chr); free_avail(lig_stack); goto big_switch;
end;
main_i:=char_info(main_f)(cur_l);
if not char_exists(main_i) then
begin char_warning(main_f,cur_chr); free_avail(lig_stack); goto big_switch;
end;
tail_append(lig_stack) {|main_loop_lookahead| is next}
@ Here we are at |main_loop_move_lig|.
When we begin this code we have |cur_l=character(lig_stack)| and |cur_q=tail|.
@<Move the cursor past a pseudo-ligature...@>=
main_p:=lig_ptr(lig_stack);
if main_p>null then tail_append(main_p);
temp_ptr:=lig_stack; lig_stack:=link(temp_ptr);
free_node(temp_ptr,small_node_size);
main_i:=char_info(main_f)(cur_l); ligature_present:=true;
if lig_stack=null then
if main_p>null then goto main_loop_lookahead
else cur_r:=bchar
else cur_r:=character(lig_stack);
goto main_lig_loop
@ The result of \.{\\char} can participate in a ligature or kern, so we must
look ahead for it.
@<Look ahead for another character...@>=
get_next; {set only |cur_cmd| and |cur_chr|, for speed}
if cur_cmd=letter then goto main_loop_lookahead+1;
if cur_cmd=other_char then goto main_loop_lookahead+1;
if cur_cmd=char_given then goto main_loop_lookahead+1;
x_token; {now expand and set |cur_cmd|, |cur_chr|, |cur_tok|}
if cur_cmd=letter then goto main_loop_lookahead+1;
if cur_cmd=other_char then goto main_loop_lookahead+1;
if cur_cmd=char_given then goto main_loop_lookahead+1;
if cur_cmd=char_num then
begin scan_char_num; cur_chr:=cur_val; goto main_loop_lookahead+1;
end;
if cur_cmd=no_boundary then bchar:=non_char;
cur_r:=bchar; lig_stack:=null; goto main_lig_loop;
main_loop_lookahead+1: adjust_space_factor;
fast_get_avail(lig_stack); font(lig_stack):=main_f;
cur_r:=qi(cur_chr); character(lig_stack):=cur_r;
if cur_r=false_bchar then cur_r:=non_char {this prevents spurious ligatures}
@ Even though comparatively few characters have a lig/kern program, several
of the instructions here count as part of \TeX's inner loop, since a
potentially long sequential search must be performed. For example, tests with
Computer Modern Roman showed that about 40 per cent of all characters
actually encountered in practice had a lig/kern program, and that about four
lig/kern commands were investigated for every such character.
At the beginning of this code we have |main_i=char_info(main_f)(cur_l)|.
@<If there's a ligature/kern command...@>=
if char_tag(main_i)<>lig_tag then goto main_loop_wrapup;
main_k:=lig_kern_start(main_f)(main_i); main_j:=font_info[main_k].qqqq;
if skip_byte(main_j)<=stop_flag then goto main_lig_loop+2;
main_k:=lig_kern_restart(main_f)(main_j);
main_lig_loop+1:main_j:=font_info[main_k].qqqq;
main_lig_loop+2:if next_char(main_j)=cur_r then
if skip_byte(main_j)<=stop_flag then
@<Do ligature or kern command, returning to |main_lig_loop|
or |main_loop_wrapup| or |main_loop_move|@>;
if skip_byte(main_j)=qi(0) then incr(main_k)
else begin if skip_byte(main_j)>=stop_flag then goto main_loop_wrapup;
main_k:=main_k+qo(skip_byte(main_j))+1;
end;
goto main_lig_loop+1
@ When a ligature or kern instruction matches a character, we know from
|read_font_info| that the character exists in the font, even though we
haven't verified its existence in the normal way.
This section could be made into a subroutine, if the code inside
|main_control| needs to be shortened.
\chardef\?='174 % vertical line to indicate character retention
@<Do ligature or kern command...@>=
begin if op_byte(main_j)>=kern_flag then
begin wrapup(rt_hit);
tail_append(new_kern(char_kern(main_f)(main_j))); goto main_loop_move;
end;
if cur_l=non_char then lft_hit:=true
else if lig_stack=null then rt_hit:=true;
check_interrupt; {allow a way out in case there's an infinite ligature loop}
case op_byte(main_j) of
qi(1),qi(5):begin cur_l:=rem_byte(main_j); {\.{=:\?}, \.{=:\?>}}
main_i:=char_info(main_f)(cur_l); ligature_present:=true;
end;
qi(2),qi(6):begin cur_r:=rem_byte(main_j); {\.{\?=:}, \.{\?=:>}}
if lig_stack=null then {right boundary character is being consumed}
begin lig_stack:=new_lig_item(cur_r); bchar:=non_char;
end
else if is_char_node(lig_stack) then {|link(lig_stack)=null|}
begin main_p:=lig_stack; lig_stack:=new_lig_item(cur_r);
lig_ptr(lig_stack):=main_p;
end
else character(lig_stack):=cur_r;
end;
qi(3):begin cur_r:=rem_byte(main_j); {\.{\?=:\?}}
main_p:=lig_stack; lig_stack:=new_lig_item(cur_r);
link(lig_stack):=main_p;
end;
qi(7),qi(11):begin wrapup(false); {\.{\?=:\?>}, \.{\?=:\?>>}}
cur_q:=tail; cur_l:=rem_byte(main_j);
main_i:=char_info(main_f)(cur_l); ligature_present:=true;
end;
othercases begin cur_l:=rem_byte(main_j); ligature_present:=true; {\.{=:}}
if lig_stack=null then goto main_loop_wrapup
else goto main_loop_move+1;
end
endcases;
if op_byte(main_j)>qi(4) then
if op_byte(main_j)<>qi(7) then goto main_loop_wrapup;
if cur_l<non_char then goto main_lig_loop;
main_k:=bchar_label[main_f]; goto main_lig_loop+1;
end
@z
% HINTS for making main_control shorter if you need to conserve space:
% (1) make pack_lig(#) a procedure call instead of a macro call
% (2) make the code of @<Do ligature...@> a function that returns one of
% four values; you goto a label based on the value returned.
@x module 1042
begin p:=font_glue[cur_font];
if p=null then
begin f:=cur_font; p:=new_spec(zero_glue); k:=param_base[f]+space_code;
width(p):=font_info[k].sc; {that's |space(f)|}
stretch(p):=font_info[k+1].sc; {and |space_stretch(f)|}
shrink(p):=font_info[k+2].sc; {and |space_shrink(f)|}
font_glue[f]:=p;
@y
begin main_p:=font_glue[cur_font];
if main_p=null then
begin main_p:=new_spec(zero_glue); main_k:=param_base[cur_font]+space_code;
width(main_p):=font_info[main_k].sc; {that's |space(cur_font)|}
stretch(main_p):=font_info[main_k+1].sc; {and |space_stretch(cur_font)|}
shrink(main_p):=font_info[main_k+2].sc; {and |space_shrink(cur_font)|}
font_glue[cur_font]:=main_p;
@z
@x module 1045
any_mode(relax),vmode+spacer,mmode+spacer,mmode+no_boundary:do_nothing;
@y
any_mode(relax),vmode+spacer,mmode+spacer:do_nothing;
@z
@x module 1090
vmode+ex_space:@t@>@;@/
@y
vmode+ex_space,vmode+no_boundary:@t@>@;@/
@z
@x module 1322
dump_int(font_glue[k]);@/
@y
dump_int(font_glue[k]);@/
dump_int(bchar_label[k]);
dump_int(font_bchar[k]);
dump_int(font_false_bchar[k]);@/
@z
@x module 1323
undump(min_halfword)(lo_mem_max)(font_glue[k]);@/
@y
undump(min_halfword)(lo_mem_max)(font_glue[k]);@/
undump(0)(font_mem_size)(bchar_label[k]);
undump(min_quarterword)(non_char)(font_bchar[k]);
undump(min_quarterword)(non_char)(font_false_bchar[k]);
@z
363. New \inputlineno feature desired by Spivak
@x module 416 gets a new definition and a new statement
@y
@d input_line_no_code=glue_val+1 {code for \.{\\inputlineno}}
primitive("inputlineno",last_item,input_line_no_code);
@!@:input_line_no_}{\.{\\inputlineno} primitive@>
@z and module 417 changes in the obvious way
@x module 424 gets new code at the beginning
@y
if cur_chr>glue_val then
begin cur_val:=line; cur_val_level:=int_val;
end
else
@z
364. New feature \holdinginserts suggested by Mittelbach.
@x modules 236--238 get a new integer parameter
@d int_pars=53 {total number of integer parameters}
@y
@d holding_inserts_code=53 {do not remove insertion nodes from \.{\\box255}}
@d int_pars=54 {total number of integer parameters}
@z and appropriate further lines are added to match all the other parameters
@x module 1014
@<Prepare all the boxes involved in insertions to act as queues@>;
q:=hold_head; link(q):=null; prev_p:=page_head; p:=link(prev_p);
while p<>best_page_break do
begin if type(p)=ins_node then @<Either insert the material
specified by node |p| into the appropriate box, or
hold it for the next page; also delete node |p| from
the current page@>
@y
if holding_inserts<=0 then
@<Prepare all the boxes involved in insertions to act as queues@>;
q:=hold_head; link(q):=null; prev_p:=page_head; p:=link(prev_p);
while p<>best_page_break do
begin if type(p)=ins_node then
begin if holding_inserts<=0 then
@<Either insert the material specified by node |p| into the
appropriate box, or hold it for the next page;
also delete node |p| from the current page@>;
end
@z
% also insert begin...end around the Pascal code of module 1018
365. New \badness feature (which I'd been resisting for years)
@x module 416 gets a new definition and a new statement
@y
@d badness_code=glue_val+2 {code for \.{\\badness}}
primitive("badness",last_item,badness_code);
@!@:badness_}{\.{\\badness} primitive@>
@z and module 417 changes in the obvious way
@x module 424 (the new code just added in #363)
begin cur_val:=line; cur_val_level:=int_val;
@y
begin if cur_chr=input_line_no_code then cur_val:=line
else cur_val:=last_badness; {|cur_chr=badness_code|}
cur_val_level:=int_val;
@z
@x module 646 gets a new global variable
@y
@!last_badness:integer; {badness of the most recently packaged box}
@z
@x module 648
@ @<Set init...@>=adjust_tail:=null;
@y
@ @<Set init...@>=adjust_tail:=null; last_badness:=0;
@z
@x modules 649 and 668
@!b:integer; {badness of the new box}
begin r:=get_node(box_node_size); type(r):=hlist_node;
@y (except it's vlist_node in 668)
begin last_badness:=0; r:=get_node(box_node_size); type(r):=hlist_node;
@z (except it's vlist_node in 668)
% change b to last_badness in modules 660&674 (4 times), 667&678 (3 times)
@x modules 658 and 673
if (hbadness<inf_bad)and(o=normal)and(list_ptr(r)<>null) then
@y (except it's vbadness in 673)
if o=normal then if list_ptr(r)<>null then
@z
@x modules 664 and 676
begin set_glue_ratio_one(glue_set(r)); {this is the maximum shrinkage}
@y
begin last_badness:=1000000;
set_glue_ratio_one(glue_set(r)); {use the maximum shrinkage}
@z
@x ibid
else if (hbadness<100)and(o=normal)and(list_ptr(r)<>null) then
@y (except it's vbadness in 676)
else if o=normal then if list_ptr(r)<>null then
@z
366. New \emergencystretch feature
@x modules 247--248 get a new dimen parameter
@d dimen_pars=20 {total number of dimension parameters}
@y
@d emergency_stretch_code=20 {reduces badnesses on final pass of line-breaking}
@d dimen_pars=21 {total number of dimension parameters}
@z and appropriate further lines are added to match all the other parameters
@x module 828
@!second_pass:boolean; {is this our second attempt to break this paragraph?}
@y
@!second_pass:boolean; {is this our second attempt to break this paragraph?}
@!final_pass:boolean; {is this our final attempt to break this paragraph?}
@z
% change second_pass to final_pass in modules 854 and 873
@x module 863
second_pass:=false;
end
else begin threshold:=tolerance; second_pass:=true;
@y
second_pass:=false; final_pass:=false;
end
else begin threshold:=tolerance; second_pass:=true;
final_pass:=(emergency_stretch<=0);
@z
@x ibid
@!stat if tracing_paragraphs>0 then print_nl("@@secondpass");@;@+tats@/
threshold:=tolerance; second_pass:=true; {if at first you don't
succeed, \dots}
@y
if not second_pass then
begin@!stat if tracing_paragraphs>0 then print_nl("@@secondpass");@;@+tats@/
threshold:=tolerance; second_pass:=true; final_pass:=(emergency_stretch<=0);
end {if at first you don't succeed, \dots}
else begin @!stat if tracing_paragraphs>0 then
print_nl("@@finalpass");@;@+tats@/
background[2]:=background[2]+emergency_stretch; final_pass:=true;
end;
@z
367. New \errorcontextlines feature suggested by a TUG participant
@x modules 236--238 get a new integer parameter
@d int_pars=54 {total number of integer parameters}
@y
@d error_context_lines_code=54 {maximum intermediate line pairs shown}
@d int_pars=55 {total number of integer parameters}
@z and appropriate further lines are added to match all the other parameters
@x module 311
@<Local variables for formatting calculations@>@/
begin base_ptr:=input_ptr; input_stack[base_ptr]:=cur_input;
{store current state}
loop@+begin cur_input:=input_stack[base_ptr]; {enter into the context}
@<Display the current context@>;
if (state<>token_list) then
if (name>17) or (base_ptr=0) then goto done;
@y
@!nn:integer; {number of contexts shown so far, less one}
@!bottom_line:boolean; {have we reached the final context to be shown?}
@<Local variables for formatting calculations@>@/
begin base_ptr:=input_ptr; input_stack[base_ptr]:=cur_input;
{store current state}
nn:=-1; bottom_line:=false;
loop@+begin cur_input:=input_stack[base_ptr]; {enter into the context}
if (state<>token_list) then
if (name>17) or (base_ptr=0) then bottom_line:=true;
if (base_ptr=input_ptr)or bottom_line or(nn<error_context_lines) then
@<Display the current context@>
else if nn=error_context_lines then
begin print_nl("..."); incr(nn); {omitted if |error_context_lines<0|}
end;
if bottom_line then goto done;
@z
% also insert begin...end around the Pascal code of module 312
% and add the statement "incr(nn)" to that module
368. char_warning inside hyphenation could clobber old_setting
@x module 863
done: @!stat if tracing_paragraphs>0 then end_diagnostic(true);@;@+tats@/
@y
done: @!stat if tracing_paragraphs>0 then
begin end_diagnostic(true); normalize_selector;
end;@+tats@/
@z
369. Make ".fmt" more easily switchable (Don Hosek).
@x module 520 gets a new definition
@y
@d format_extension=".fmt" {the extension, as a \.{WEB} constant}
@z now replace ".fmt" by format_extension in modules 529 and 1328.
370. Possible range check on weird nullfont (Breitenlohner, 16 Oct 89).
@x module 565
if (bc>ec+1)or(ec>255) then abort;
@y
if (bc>ec+1)or(ec>255) then abort;
if bc>255 then {|bc=256| and |ec=255|}
begin bc:=1; ec:=0;
end;
@z
371. (I sincerely hope that there won't be any more)
* Possibly nice ideas that will not be implemented
. Classes of marks analogous to classes of insertions
. \showcontext to show the current location without stopping for error
. \everyeof to insert tokens before an \input file ends
(strange example: \everyeof{\noexpand} will allow things
like \xdef\a{\input foo}!)
. generalize \leftskip and \rightskip to token lists (problems with
displayed math then)
. generalize \widowline and \clubline to go further into a paragraph
* Bad ideas that will not be implemented
. several people want to be able to remove arbitrary elements of lists,
but that must never be done because some of those elements (e.g. kerns
for accents) depend on floating point arithmetic
. if anybody wants letter spacing desperately they should put it in their own
private version (e.g. generalize the hpack routine) and NOT call it TeX.