diff options
Diffstat (limited to 'WebKitTools/android/flex-2.5.4a/NEWS')
| -rw-r--r-- | WebKitTools/android/flex-2.5.4a/NEWS | 1233 |
1 files changed, 0 insertions, 1233 deletions
diff --git a/WebKitTools/android/flex-2.5.4a/NEWS b/WebKitTools/android/flex-2.5.4a/NEWS deleted file mode 100644 index 3e23e7d..0000000 --- a/WebKitTools/android/flex-2.5.4a/NEWS +++ /dev/null @@ -1,1233 +0,0 @@ -Changes between release 2.5.4 (11Sep96) and release 2.5.3: - - - Fixed a bug introduced in 2.5.3 that blew it when a call - to input() occurred at the end of an input file. - - - Fixed scanner skeleton so the example in the man page of - scanning strings using exclusive start conditions works. - - - Minor Makefile tweaks. - - -Changes between release 2.5.3 (29May96) and release 2.5.2: - - - Some serious bugs in yymore() have been fixed. In particular, - when using AT&T-lex-compatibility or %array, you can intermix - calls to input(), unput(), and yymore(). (This still doesn't - work for %pointer, and isn't likely to in the future.) - - - A bug in handling NUL's in the input stream of scanners using - REJECT has been fixed. - - - The default main() in libfl.a now repeatedly calls yylex() until - it returns 0, rather than just calling it once. - - - Minor tweak for Windows NT Makefile, MISC/NT/Makefile. - - -Changes between release 2.5.2 (25Apr95) and release 2.5.1: - - - The --prefix configuration option now works. - - - A bug that completely broke the "-Cf" table compression - option has been fixed. - - - A major headache involving "const" declarators and Solaris - systems has been fixed. - - - An octal escape sequence in a flex regular expression must - now contain only the digits 0-7. - - - You can now use "--" on the flex command line to mark the - end of flex options. - - - You can now specify the filename '-' as a synonym for stdin. - - - By default, the scanners generated by flex no longer - statically initialize yyin and yyout to stdin and stdout. - This change is necessary because in some ANSI environments, - stdin and stdout are not compile-time constant. You can - force the initialization using "%option stdinit" in the first - section of your flex input. - - - "%option nounput" now correctly omits the unput() routine - from the output. - - - "make clean" now removes config.log, config.cache, and the - flex binary. The fact that it removes the flex binary means - you should take care if making changes to scan.l, to make - sure you don't wind up in a bootstrap problem. - - - In general, the Makefile has been reworked somewhat (thanks - to Francois Pinard) for added flexibility - more changes will - follow in subsequent releases. - - - The .texi and .info files in MISC/texinfo/ have been updated, - thanks also to Francois Pinard. - - - The FlexLexer::yylex(istream* new_in, ostream* new_out) method - now does not have a default for the first argument, to disambiguate - it from FlexLexer::yylex(). - - - A bug in destructing a FlexLexer object before doing any scanning - with it has been fixed. - - - A problem with including FlexLexer.h multiple times has been fixed. - - - The alloca() chud necessary to accommodate bison has grown - even uglier, but hopefully more correct. - - - A portability tweak has been added to accommodate compilers that - use char* generic pointers. - - - EBCDIC contact information in the file MISC/EBCDIC has been updated. - - - An OS/2 Makefile and config.h for flex 2.5 is now available in - MISC/OS2/, contributed by Kai Uwe Rommel. - - - The descrip.mms file for building flex under VMS has been updated, - thanks to Pat Rankin. - - - The notes on building flex for the Amiga have been updated for - flex 2.5, contributed by Andreas Scherer. - - -Changes between release 2.5.1 (28Mar95) and release 2.4.7: - - - A new concept of "start condition" scope has been introduced. - A start condition scope is begun with: - - <SCs>{ - - where SCs is a list of one or more start conditions. Inside - the start condition scope, every rule automatically has the - prefix <SCs> applied to it, until a '}' which matches the - initial '{'. So, for example: - - <ESC>{ - "\\n" return '\n'; - "\\r" return '\r'; - "\\f" return '\f'; - "\\0" return '\0'; - } - - is equivalent to: - - <ESC>"\\n" return '\n'; - <ESC>"\\r" return '\r'; - <ESC>"\\f" return '\f'; - <ESC>"\\0" return '\0'; - - As indicated in this example, rules inside start condition scopes - (and any rule, actually, other than the first) can be indented, - to better show the extent of the scope. - - Start condition scopes may be nested. - - - The new %option directive can be used in the first section of - a flex scanner to control scanner-generation options. Most - options are given simply as names, optionally preceded by the - word "no" (with no intervening whitespace) to negate their - meaning. Some are equivalent to flex flags, so putting them - in your scanner source is equivalent to always specifying - the flag (%option's take precedence over flags): - - 7bit -7 option - 8bit -8 option - align -Ca option - backup -b option - batch -B option - c++ -+ option - caseful opposite of -i option (caseful is the default); - case-sensitive same as above - caseless -i option; - case-insensitive same as above - debug -d option - default opposite of -s option - ecs -Ce option - fast -F option - full -f option - interactive -I option - lex-compat -l option - meta-ecs -Cm option - perf-report -p option - read -Cr option - stdout -t option - verbose -v option - warn opposite of -w option (so use "%option nowarn" for -w) - - array equivalent to "%array" - pointer equivalent to "%pointer" (default) - - Some provide new features: - - always-interactive generate a scanner which always - considers its input "interactive" (no call to isatty() - will be made when the scanner runs) - main supply a main program for the scanner, which - simply calls yylex(). Implies %option noyywrap. - never-interactive generate a scanner which never - considers its input "interactive" (no call to isatty() - will be made when the scanner runs) - stack if set, enable start condition stacks (see below) - stdinit if unset ("%option nostdinit"), initialize yyin - and yyout statically to nil FILE* pointers, instead - of stdin and stdout - yylineno if set, keep track of the current line - number in global yylineno (this option is expensive - in terms of performance). The line number is available - to C++ scanning objects via the new member function - lineno(). - yywrap if unset ("%option noyywrap"), scanner does not - call yywrap() upon EOF but simply assumes there - are no more files to scan - - Flex scans your rule actions to determine whether you use the - REJECT or yymore features (this is not new). Two %options can be - used to override its decision, either by setting them to indicate - the feature is indeed used, or unsetting them to indicate it - actually is not used: - - reject - yymore - - Three %option's take string-delimited values, offset with '=': - - outfile="<name>" equivalent to -o<name> - prefix="<name>" equivalent to -P<name> - yyclass="<name>" set the name of the C++ scanning class - (see below) - - A number of %option's are available for lint purists who - want to suppress the appearance of unneeded routines in - the generated scanner. Each of the following, if unset, - results in the corresponding routine not appearing in the - generated scanner: - - input, unput - yy_push_state, yy_pop_state, yy_top_state - yy_scan_buffer, yy_scan_bytes, yy_scan_string - - You can specify multiple options with a single %option directive, - and multiple directives in the first section of your flex input file. - - - The new function: - - YY_BUFFER_STATE yy_scan_string( const char *str ) - - returns a YY_BUFFER_STATE (which also becomes the current input - buffer) for scanning the given string, which occurs starting - with the next call to yylex(). The string must be NUL-terminated. - A related function: - - YY_BUFFER_STATE yy_scan_bytes( const char *bytes, int len ) - - creates a buffer for scanning "len" bytes (including possibly NUL's) - starting at location "bytes". - - Note that both of these functions create and scan a *copy* of - the string/bytes. (This may be desirable, since yylex() modifies - the contents of the buffer it is scanning.) You can avoid the - copy by using: - - YY_BUFFER_STATE yy_scan_buffer( char *base, yy_size_t size ) - - which scans in place the buffer starting at "base", consisting - of "size" bytes, the last two bytes of which *must* be - YY_END_OF_BUFFER_CHAR (these bytes are not scanned; thus, scanning - consists of base[0] through base[size-2], inclusive). If you - fail to set up "base" in this manner, yy_scan_buffer returns a - nil pointer instead of creating a new input buffer. - - The type yy_size_t is an integral type to which you can cast - an integer expression reflecting the size of the buffer. - - - Three new routines are available for manipulating stacks of - start conditions: - - void yy_push_state( int new_state ) - - pushes the current start condition onto the top of the stack - and BEGIN's "new_state" (recall that start condition names are - also integers). - - void yy_pop_state() - - pops the top of the stack and BEGIN's to it, and - - int yy_top_state() - - returns the top of the stack without altering the stack's - contents. - - The start condition stack grows dynamically and so has no built-in - size limitation. If memory is exhausted, program execution - is aborted. - - To use start condition stacks, your scanner must include - a "%option stack" directive. - - - flex now supports POSIX character class expressions. These - are expressions enclosed inside "[:" and ":]" delimiters (which - themselves must appear between the '[' and ']' of a character - class; other elements may occur inside the character class, too). - The expressions flex recognizes are: - - [:alnum:] [:alpha:] [:blank:] [:cntrl:] [:digit:] [:graph:] - [:lower:] [:print:] [:punct:] [:space:] [:upper:] [:xdigit:] - - These expressions all designate a set of characters equivalent to - the corresponding isXXX function (for example, [:alnum:] designates - those characters for which isalnum() returns true - i.e., any - alphabetic or numeric). Some systems don't provide isblank(), - so flex defines [:blank:] as a blank or a tab. - - For example, the following character classes are all equivalent: - - [[:alnum:]] - [[:alpha:][:digit:] - [[:alpha:]0-9] - [a-zA-Z0-9] - - If your scanner is case-insensitive (-i flag), then [:upper:] - and [:lower:] are equivalent to [:alpha:]. - - - The promised rewrite of the C++ FlexLexer class has not yet - been done. Support for FlexLexer is limited at the moment to - fixing show-stopper bugs, so, for example, the new functions - yy_scan_string() & friends are not available to FlexLexer - objects. - - - The new macro - - yy_set_interactive(is_interactive) - - can be used to control whether the current buffer is considered - "interactive". An interactive buffer is processed more slowly, - but must be used when the scanner's input source is indeed - interactive to avoid problems due to waiting to fill buffers - (see the discussion of the -I flag in flex.1). A non-zero value - in the macro invocation marks the buffer as interactive, a zero - value as non-interactive. Note that use of this macro overrides - "%option always-interactive" or "%option never-interactive". - - yy_set_interactive() must be invoked prior to beginning to - scan the buffer. - - - The new macro - - yy_set_bol(at_bol) - - can be used to control whether the current buffer's scanning - context for the next token match is done as though at the - beginning of a line (non-zero macro argument; makes '^' anchored - rules active) or not at the beginning of a line (zero argument, - '^' rules inactive). - - - Related to this change, the mechanism for determining when a scan is - starting at the beginning of a line has changed. It used to be - that '^' was active iff the character prior to that at which the - scan started was a newline. The mechanism now is that '^' is - active iff the last token ended in a newline (or the last call to - input() returned a newline). For most users, the difference in - mechanisms is negligible. Where it will make a difference, - however, is if unput() or yyless() is used to alter the input - stream. When in doubt, use yy_set_bol(). - - - The new beginning-of-line mechanism involved changing some fairly - twisted code, so it may have introduced bugs - beware ... - - - The macro YY_AT_BOL() returns true if the next token scanned from - the current buffer will have '^' rules active, false otherwise. - - - The new function - - void yy_flush_buffer( struct yy_buffer_state* b ) - - flushes the contents of the current buffer (i.e., next time - the scanner attempts to match a token using b as the current - buffer, it will begin by invoking YY_INPUT to fill the buffer). - This routine is also available to C++ scanners (unlike some - of the other new routines). - - The related macro - - YY_FLUSH_BUFFER - - flushes the contents of the current buffer. - - - A new "-ooutput" option writes the generated scanner to "output". - If used with -t, the scanner is still written to stdout, but - its internal #line directives (see previous item) use "output". - - - Flex now generates #line directives relating the code it - produces to the output file; this means that error messages - in the flex-generated code should be correctly pinpointed. - - - When generating #line directives, filenames with embedded '\'s - have those characters escaped (i.e., turned into '\\'). This - feature helps with reporting filenames for some MS-DOS and OS/2 - systems. - - - The FlexLexer class includes two new public member functions: - - virtual void switch_streams( istream* new_in = 0, - ostream* new_out = 0 ) - - reassigns yyin to new_in (if non-nil) and yyout to new_out - (ditto), deleting the previous input buffer if yyin is - reassigned. It is used by: - - int yylex( istream* new_in = 0, ostream* new_out = 0 ) - - which first calls switch_streams() and then returns the value - of calling yylex(). - - - C++ scanners now have yy_flex_debug as a member variable of - FlexLexer rather than a global, and member functions for testing - and setting it. - - - When generating a C++ scanning class, you can now use - - %option yyclass="foo" - - to inform flex that you have derived "foo" as a subclass of - yyFlexLexer, so flex will place your actions in the member - function foo::yylex() instead of yyFlexLexer::yylex(). It also - generates a yyFlexLexer::yylex() member function that generates a - run-time error if called (by invoking yyFlexLexer::LexerError()). - This feature is necessary if your subclass "foo" introduces some - additional member functions or variables that you need to access - from yylex(). - - - Current texinfo files in MISC/texinfo, contributed by Francois - Pinard. - - - You can now change the name "flex" to something else (e.g., "lex") - by redefining $(FLEX) in the Makefile. - - - Two bugs (one serious) that could cause "bigcheck" to fail have - been fixed. - - - A number of portability/configuration changes have been made - for easier portability. - - - You can use "YYSTATE" in your scanner as an alias for YY_START - (for AT&T lex compatibility). - - - input() now maintains yylineno. - - - input() no longer trashes yytext. - - - interactive scanners now read characters in YY_INPUT up to a - newline, a large performance gain. - - - C++ scanner objects now work with the -P option. You include - <FlexLexer.h> once per scanner - see comments in <FlexLexer.h> - (or flex.1) for details. - - - C++ FlexLexer objects now use the "cerr" stream to report -d output - instead of stdio. - - - The -c flag now has its full glorious POSIX interpretation (do - nothing), rather than being interpreted as an old-style -C flag. - - - Scanners generated by flex now include two #define's giving - the major and minor version numbers (YY_FLEX_MAJOR_VERSION, - YY_FLEX_MINOR_VERSION). These can then be tested to see - whether certain flex features are available. - - - Scanners generated using -l lex compatibility now have the symbol - YY_FLEX_LEX_COMPAT #define'd. - - - When initializing (i.e., yy_init is non-zero on entry to yylex()), - generated scanners now set yy_init to zero before executing - YY_USER_INIT. This means that you can set yy_init back to a - non-zero value in YY_USER_INIT if you need the scanner to be - reinitialized on the next call. - - - You can now use "#line" directives in the first section of your - scanner specification. - - - When generating full-table scanners (-Cf), flex now puts braces - around each row of the 2-d array initialization, to silence warnings - on over-zealous compilers. - - - Improved support for MS-DOS. The flex sources have been successfully - built, unmodified, for Borland 4.02 (all that's required is a - Borland Makefile and config.h file, which are supplied in - MISC/Borland - contributed by Terrence O Kane). - - - Improved support for Macintosh using Think C - the sources should - build for this platform "out of the box". Contributed by Scott - Hofmann. - - - Improved support for VMS, in MISC/VMS/, contributed by Pat Rankin. - - - Support for the Amiga, in MISC/Amiga/, contributed by Andreas - Scherer. Note that the contributed files were developed for - flex 2.4 and have not been tested with flex 2.5. - - - Some notes on support for the NeXT, in MISC/NeXT, contributed - by Raf Schietekat. - - - The MISC/ directory now includes a preformatted version of flex.1 - in flex.man, and pre-yacc'd versions of parse.y in parse.{c,h}. - - - The flex.1 and flexdoc.1 manual pages have been merged. There - is now just one document, flex.1, which includes an overview - at the beginning to help you find the section you need. - - - Documentation now clarifies that start conditions persist across - switches to new input files or different input buffers. If you - want to e.g., return to INITIAL, you must explicitly do so. - - - The "Performance Considerations" section of the manual has been - updated. - - - Documented the "yy_act" variable, which when YY_USER_ACTION is - invoked holds the number of the matched rule, and added an - example of using yy_act to profile how often each rule is matched. - - - Added YY_NUM_RULES, a definition that gives the total number - of rules in the file, including the default rule (even if you - use -s). - - - Documentation now clarifies that you can pass a nil FILE* pointer - to yy_create_buffer() or yyrestart() if you've arrange YY_INPUT - to not need yyin. - - - Documentation now clarifies that YY_BUFFER_STATE is a pointer to - an opaque "struct yy_buffer_state". - - - Documentation now stresses that you gain the benefits of removing - backing-up states only if you remove *all* of them. - - - Documentation now points out that traditional lex allows you - to put the action on a separate line from the rule pattern if - the pattern has trailing whitespace (ugh!), but flex doesn't - support this. - - - A broken example in documentation of the difference between - inclusive and exclusive start conditions is now fixed. - - - Usage (-h) report now goes to stdout. - - - Version (-V) info now goes to stdout. - - - More #ifdef chud has been added to the parser in attempt to - deal with bison's use of alloca(). - - - "make clean" no longer deletes emacs backup files (*~). - - - Some memory leaks have been fixed. - - - A bug was fixed in which dynamically-expanded buffers were - reallocated a couple of bytes too small. - - - A bug was fixed which could cause flex to read and write beyond - the end of the input buffer. - - - -S will not be going away. - - -Changes between release 2.4.7 (03Aug94) and release 2.4.6: - - - Fixed serious bug in reading multiple files. - - - Fixed bug in scanning NUL's. - - - Fixed bug in input() returning 8-bit characters. - - - Fixed bug in matching text with embedded NUL's when - using %array or lex compatibility. - - - Fixed multiple invocations of YY_USER_ACTION when using '|' - continuation action. - - - Minor prototyping fixes. - -Changes between release 2.4.6 (04Jan94) and release 2.4.5: - - - Linking with -lfl no longer required if your program includes - its own yywrap() and main() functions. (This change will cause - problems if you have a non-ANSI compiler on a system for which - sizeof(int) != sizeof(void*) or sizeof(int) != sizeof(size_t).) - - - The use of 'extern "C++"' in FlexLexer.h has been modified to - get around an incompatibility with g++'s header files. - -Changes between release 2.4.5 (11Dec93) and release 2.4.4: - - - Fixed bug breaking C++ scanners that use REJECT or variable - trailing context. - - - Fixed serious input problem for interactive scanners on - systems for which char is unsigned. - - - Fixed bug in incorrectly treating '$' operator as variable - trailing context. - - - Fixed bug in -CF table representation that could lead to - corrupt tables. - - - Fixed fairly benign memory leak. - - - Added `extern "C++"' wrapper to FlexLexer.h header. This - should overcome the g++ 2.5.X problems mentioned in the - NEWS for release 2.4.3. - - - Changed #include of FlexLexer.h to use <> instead of "". - - - Added feature to control whether the scanner attempts to - refill the input buffer once it's exhausted. This feature - will be documented in the 2.5 release. - - -Changes between release 2.4.4 (07Dec93) and release 2.4.3: - - - Fixed two serious bugs in scanning 8-bit characters. - - - Fixed bug in YY_USER_ACTION that caused it to be executed - inappropriately (on the scanner's own internal actions, and - with incorrect yytext/yyleng values). - - - Fixed bug in pointing yyin at a new file and resuming scanning. - - - Portability fix regarding min/max/abs macros conflicting with - function definitions in standard header files. - - - Added a virtual LexerError() method to the C++ yyFlexLexer class - for reporting error messages instead of always using cerr. - - - Added warning in flexdoc that the C++ scanning class is presently - experimental and subject to considerable change between major - releases. - - -Changes between release 2.4.3 (03Dec93) and release 2.4.2: - - - Fixed bug causing fatal scanner messages to fail to print. - - - Fixed things so FlexLexer.h can be included in other C++ - sources. One side-effect of this change is that -+ and -CF - are now incompatible. - - - libfl.a now supplies private versions of the the <string.h>/ - <strings.h> string routines needed by flex and the scanners - it generates, to enhance portability to some BSD systems. - - - More robust solution to 2.4.2's flexfatal() bug fix. - - - Added ranlib of installed libfl.a. - - - Some lint tweaks. - - - NOTE: problems have been encountered attempting to build flex - C++ scanners using g++ version 2.5.X. The problem is due to an - unfortunate heuristic in g++ 2.5.X that attempts to discern between - C and C++ headers. Because FlexLexer.h is installed (by default) - in /usr/local/include and not /usr/local/lib/g++-include, g++ 2.5.X - decides that it's a C header :-(. So if you have problems, install - the header in /usr/local/lib/g++-include instead. - - -Changes between release 2.4.2 (01Dec93) and release 2.4.1: - - - Fixed bug in libfl.a referring to non-existent "flexfatal" function. - - - Modified to produce both compress'd and gzip'd tar files for - distributions (you probably don't care about this change!). - - -Changes between release 2.4.1 (30Nov93) and release 2.3.8: - - - The new '-+' flag instructs flex to generate a C++ scanner class - (thanks to Kent Williams). flex writes an implementation of the - class defined in FlexLexer.h to lex.yy.cc. You may include - multiple scanner classes in your program using the -P flag. Note - that the scanner class also provides a mechanism for creating - reentrant scanners. The scanner class uses C++ streams for I/O - instead of FILE*'s (thanks to Tom Epperly). If the flex executable's - name ends in '+' then the '-+' flag is automatically on, so creating - a symlink or copy of "flex" to "flex++" results in a version of - flex that can be used exclusively for C++ scanners. - - Note that without the '-+' flag, flex-generated scanners can still - be compiled using C++ compilers, though they use FILE*'s for I/O - instead of streams. - - See the "GENERATING C++ SCANNERS" section of flexdoc for details. - - - The new '-l' flag turns on maximum AT&T lex compatibility. In - particular, -l includes support for "yylineno" and makes yytext - be an array instead of a pointer. It does not, however, do away - with all incompatibilities. See the "INCOMPATIBILITIES WITH LEX - AND POSIX" section of flexdoc for details. - - - The new '-P' option specifies a prefix to use other than "yy" - for the scanner's globally-visible variables, and for the - "lex.yy.c" filename. Using -P you can link together multiple - flex scanners in the same executable. - - - The distribution includes a "texinfo" version of flexdoc.1, - contributed by Roland Pesch (thanks also to Marq Kole, who - contributed another version). It has not been brought up to - date, but reflects version 2.3. See MISC/flex.texinfo. - - The flex distribution will soon include G.T. Nicol's flex - manual; he is presently bringing it up-to-date for version 2.4. - - - yywrap() is now a function, and you now *must* link flex scanners - with libfl.a. - - - Site-configuration is now done via an autoconf-generated - "configure" script contributed by Francois Pinard. - - - Scanners now use fread() (or getc(), if interactive) and not - read() for input. A new "table compression" option, -Cr, - overrides this change and causes the scanner to use read() - (because read() is a bit faster than fread()). -f and -F - are now equivalent to -Cfr and -CFr; i.e., they imply the - -Cr option. - - - In the blessed name of POSIX compliance, flex supports "%array" - and "%pointer" directives in the definitions (first) section of - the scanner specification. The former specifies that yytext - should be an array (of size YYLMAX), the latter, that it should - be a pointer. The array version of yytext is universally slower - than the pointer version, but has the advantage that its contents - remain unmodified across calls to input() and unput() (the pointer - version of yytext is, still, trashed by such calls). - - "%array" cannot be used with the '-+' C++ scanner class option. - - - The new '-Ca' option directs flex to trade off memory for - natural alignment when generating a scanner's tables. In - particular, table entries that would otherwise be "short" - become "long". - - - The new '-h' option produces a summary of the flex flags. - - - The new '-V' option reports the flex version number and exits. - - - The new scanner macro YY_START returns an integer value - corresponding to the current start condition. You can return - to that start condition by passing the value to a subsequent - "BEGIN" action. You also can implement "start condition stacks" - by storing the values in an integer stack. - - - You can now redefine macros such as YY_INPUT by just #define'ing - them to some other value in the first section of the flex input; - no need to first #undef them. - - - flex now generates warnings for rules that can't be matched. - These warnings can be turned off using the new '-w' flag. If - your scanner uses REJECT then you will not get these warnings. - - - If you specify the '-s' flag but the default rule can be matched, - flex now generates a warning. - - - "yyleng" is now a global, and may be modified by the user (though - doing so and then using yymore() will yield weird results). - - - Name definitions in the first section of a scanner specification - can now include a leading '^' or trailing '$' operator. In this - case, the definition is *not* pushed back inside of parentheses. - - - Scanners with compressed tables are now "interactive" (-I option) - by default. You can suppress this attribute (which makes them - run slightly slower) using the new '-B' flag. - - - Flex now generates 8-bit scanners by default, unless you use the - -Cf or -CF compression options (-Cfe and -CFe result in 8-bit - scanners). You can force it to generate a 7-bit scanner using - the new '-7' flag. You can build flex to generate 8-bit scanners - for -Cf and -CF, too, by adding -DDEFAULT_CSIZE=256 to CFLAGS - in the Makefile. - - - You no longer need to call the scanner routine yyrestart() to - inform the scanner that you have switched to a new file after - having seen an EOF on the current input file. Instead, just - point yyin at the new file and continue scanning. - - - You no longer need to invoke YY_NEW_FILE in an <<EOF>> action - to indicate you wish to continue scanning. Simply point yyin - at a new file. - - - A leading '#' no longer introduces a comment in a flex input. - - - flex no longer considers formfeed ('\f') a whitespace character. - - - %t, I'm happy to report, has been nuked. - - - The '-p' option may be given twice ('-pp') to instruct flex to - report minor performance problems as well as major ones. - - - The '-v' verbose output no longer includes start/finish time - information. - - - Newlines in flex inputs can optionally include leading or - trailing carriage-returns ('\r'), in support of several PC/Mac - run-time libraries that automatically include these. - - - A start condition of the form "<*>" makes the following rule - active in every start condition, whether exclusive or inclusive. - - - The following items have been corrected in the flex documentation: - - - '-C' table compression options *are* cumulative. - - - You may modify yytext but not lengthen it by appending - characters to the end. Modifying its final character - will affect '^' anchoring for the next rule matched - if the character is changed to or from a newline. - - - The term "backtracking" has been renamed "backing up", - since it is a one-time repositioning and not a repeated - search. What used to be the "lex.backtrack" file is now - "lex.backup". - - - Unindented "/* ... */" comments are allowed in the first - flex input section, but not in the second. - - - yyless() can only be used in the flex input source, not - externally. - - - You can use "yyrestart(yyin)" to throw away the - current contents of the input buffer. - - - To write high-speed scanners, attempt to match as much - text as possible with each rule. See MISC/fastwc/README - for more information. - - - Using the beginning-of-line operator ('^') is fairly - cheap. Using unput() is expensive. Using yyless() is - cheap. - - - An example of scanning strings with embedded escape - sequences has been added. - - - The example of backing-up in flexdoc was erroneous; it - has been corrected. - - - A flex scanner's internal buffer now dynamically grows if needed - to match large tokens. Note that growing the buffer presently - requires rescanning the (large) token, so consuming a lot of - text this way is a slow process. Also note that presently the - buffer does *not* grow if you unput() more text than can fit - into the buffer. - - - The MISC/ directory has been reorganized; see MISC/README for - details. - - - yyless() can now be used in the third (user action) section - of a scanner specification, thanks to Ceriel Jacobs. yyless() - remains a macro and cannot be used outside of the scanner source. - - - The skeleton file is no longer opened at run-time, but instead - compiled into a large string array (thanks to John Gilmore and - friends at Cygnus). You can still use the -S flag to point flex - at a different skeleton file. - - - flex no longer uses a temporary file to store the scanner's - actions. - - - A number of changes have been made to decrease porting headaches. - In particular, flex no longer uses memset() or ctime(), and - provides a single simple mechanism for dealing with C compilers - that still define malloc() as returning char* instead of void*. - - - Flex now detects if the scanner specification requires the -8 flag - but the flag was not given or on by default. - - - A number of table-expansion fencepost bugs have been fixed, - making flex more robust for generating large scanners. - - - flex more consistently identifies the location of errors in - its input. - - - YY_USER_ACTION is now invoked only for "real" actions, not for - internal actions used by the scanner for things like filling - the buffer or handling EOF. - - - The rule "[^]]" now matches any character other than a ']'; - formerly it matched any character at all followed by a ']'. - This change was made for compatibility with AT&T lex. - - - A large number of miscellaneous bugs have been found and fixed - thanks to Gerhard Wilhelms. - - - The source code has been heavily reformatted, making patches - relative to previous flex releases no longer accurate. - - -Changes between 2.3 Patch #8 (21Feb93) and 2.3 Patch #7: - - - Fixed bugs in dynamic memory allocation leading to grievous - fencepost problems when generating large scanners. - - Fixed bug causing infinite loops on character classes with 8-bit - characters in them. - - Fixed bug in matching repetitions with a lower bound of 0. - - Fixed bug in scanning NUL characters using an "interactive" scanner. - - Fixed bug in using yymore() at the end of a file. - - Fixed bug in misrecognizing rules with variable trailing context. - - Fixed bug compiling flex on Suns using gcc 2. - - Fixed bug in not recognizing that input files with the character - ASCII 128 in them require the -8 flag. - - Fixed bug that could cause an infinite loop writing out - error messages. - - Fixed bug in not recognizing old-style lex % declarations if - followed by a tab instead of a space. - - Fixed potential crash when flex terminated early (usually due - to a bad flag) and the -v flag had been given. - - Added some missing declarations of void functions. - - Changed to only use '\a' for __STDC__ compilers. - - Updated mailing addresses. - - -Changes between 2.3 Patch #7 (28Mar91) and 2.3 Patch #6: - - - Fixed out-of-bounds array access that caused bad tables - to be produced on machines where the bad reference happened - to yield a 1. This caused problems installing or running - flex on some Suns, in particular. - - -Changes between 2.3 Patch #6 (29Aug90) and 2.3 Patch #5: - - - Fixed a serious bug in yymore() which basically made it - completely broken. Thanks goes to Jean Christophe of - the Nethack development team for finding the problem - and passing along the fix. - - -Changes between 2.3 Patch #5 (16Aug90) and 2.3 Patch #4: - - - An up-to-date version of initscan.c so "make test" will - work after applying the previous patches - - -Changes between 2.3 Patch #4 (14Aug90) and 2.3 Patch #3: - - - Fixed bug in hexadecimal escapes which allowed only digits, - not letters, in escapes - - Fixed bug in previous "Changes" file! - - -Changes between 2.3 Patch #3 (03Aug90) and 2.3 Patch #2: - - - Correction to patch #2 for gcc compilation; thanks goes to - Paul Eggert for catching this. - - -Changes between 2.3 Patch #2 (02Aug90) and original 2.3 release: - - - Fixed (hopefully) headaches involving declaring malloc() - and free() for gcc, which defines __STDC__ but (often) doesn't - come with the standard include files such as <stdlib.h>. - Reordered #ifdef maze in the scanner skeleton in the hope of - getting the declarations right for cfront and g++, too. - - - Note that this patch supercedes patch #1 for release 2.3, - which was never announced but was available briefly for - anonymous ftp. - - -Changes between 2.3 (full) release of 28Jun90 and 2.2 (alpha) release: - - User-visible: - - - A lone <<EOF>> rule (that is, one which is not qualified with - a list of start conditions) now specifies the EOF action for - *all* start conditions which haven't already had <<EOF>> actions - given. To specify an end-of-file action for just the initial - state, use <INITIAL><<EOF>>. - - - -d debug output is now contigent on the global yy_flex_debug - being set to a non-zero value, which it is by default. - - - A new macro, YY_USER_INIT, is provided for the user to specify - initialization action to be taken on the first call to the - scanner. This action is done before the scanner does its - own initialization. - - - yy_new_buffer() has been added as an alias for yy_create_buffer() - - - Comments beginning with '#' and extending to the end of the line - now work, but have been deprecated (in anticipation of making - flex recognize #line directives). - - - The funky restrictions on when semi-colons could follow the - YY_NEW_FILE and yyless macros have been removed. They now - behave identically to functions. - - - A bug in the sample redefinition of YY_INPUT in the documentation - has been corrected. - - - A bug in the sample simple tokener in the documentation has - been corrected. - - - The documentation on the incompatibilities between flex and - lex has been reordered so that the discussion of yylineno - and input() come first, as it's anticipated that these will - be the most common source of headaches. - - - Things which didn't used to be documented but now are: - - - flex interprets "^foo|bar" differently from lex. flex interprets - it as "match either a 'foo' or a 'bar', providing it comes at the - beginning of a line", whereas lex interprets it as "match either - a 'foo' at the beginning of a line, or a 'bar' anywhere". - - - flex initializes the global "yyin" on the first call to the - scanner, while lex initializes it at compile-time. - - - yy_switch_to_buffer() can be used in the yywrap() macro/routine. - - - flex scanners do not use stdio for their input, and hence when - writing an interactive scanner one must explictly call fflush() - after writing out a prompt. - - - flex scanner can be made reentrant (after a fashion) by using - "yyrestart( yyin );". This is useful for interactive scanners - which have interrupt handlers that long-jump out of the scanner. - - - a defense of why yylineno is not supported is included, along - with a suggestion on how to convert scanners which rely on it. - - - Other changes: - - - Prototypes and proper declarations of void routines have - been added to the flex source code, courtesy of Kevin B. Kenny. - - - Routines dealing with memory allocation now use void* pointers - instead of char* - see Makefile for porting implications. - - - Error-checking is now done when flex closes a file. - - - Various lint tweaks were added to reduce the number of gripes. - - - Makefile has been further parameterized to aid in porting. - - - Support for SCO Unix added. - - - Flex now sports the latest & greatest UC copyright notice - (which is only slightly different from the previous one). - - - A note has been added to flexdoc.1 mentioning work in progress - on modifying flex to generate straight C code rather than a - table-driven automaton, with an email address of whom to contact - if you are working along similar lines. - - -Changes between 2.2 Patch #3 (30Mar90) and 2.2 Patch #2: - - - fixed bug which caused -I scanners to bomb - - -Changes between 2.2 Patch #2 (27Mar90) and 2.2 Patch #1: - - - fixed bug writing past end of input buffer in yyunput() - - fixed bug detecting NUL's at the end of a buffer - - -Changes between 2.2 Patch #1 (23Mar90) and 2.2 (alpha) release: - - - Makefile fixes: definition of MAKE variable for systems - which don't have it; installation of flexdoc.1 along with - flex.1; fixed two bugs which could cause "bigtest" to fail. - - - flex.skel fix for compiling with g++. - - - README and flexdoc.1 no longer list an out-of-date BITNET address - for contacting me. - - - minor typos and formatting changes to flex.1 and flexdoc.1. - - -Changes between 2.2 (alpha) release of March '90 and previous release: - - User-visible: - - - Full user documentation now available. - - - Support for 8-bit scanners. - - - Scanners now accept NUL's. - - - A facility has been added for dealing with multiple - input buffers. - - - Two manual entries now. One which fully describes flex - (rather than just its differences from lex), and the - other for quick(er) reference. - - - A number of changes to bring flex closer into compliance - with the latest POSIX lex draft: - - %t support - flex now accepts multiple input files and concatenates - them together to form its input - previous -c (compress) flag renamed -C - do-nothing -c and -n flags added - Any indented code or code within %{}'s in section 2 is - now copied to the output - - - yyleng is now a bona fide global integer. - - - -d debug information now gives the line number of the - matched rule instead of which number rule it was from - the beginning of the file. - - - -v output now includes a summary of the flags used to generate - the scanner. - - - unput() and yyrestart() are now globally callable. - - - yyrestart() no longer closes the previous value of yyin. - - - C++ support; generated scanners can be compiled with C++ compiler. - - - Primitive -lfl library added, containing default main() - which calls yylex(). A number of routines currently living - in the scanner skeleton will probably migrate to here - in the future (in particular, yywrap() will probably cease - to be a macro and instead be a function in the -lfl library). - - - Hexadecimal (\x) escape sequences added. - - - Support for MS-DOS, VMS, and Turbo-C integrated. - - - The %used/%unused operators have been deprecated. They - may go away soon. - - - Other changes: - - - Makefile enhanced for easier testing and installation. - - The parser has been tweaked to detect some erroneous - constructions which previously were missed. - - Scanner input buffer overflow is now detected. - - Bugs with missing "const" declarations fixed. - - Out-of-date Minix/Atari patches provided. - - Scanners no longer require printf() unless FLEX_DEBUG is being used. - - A subtle input() bug has been fixed. - - Line numbers for "continued action" rules (those following - the special '|' action) are now correct. - - unput() bug fixed; had been causing problems porting flex to VMS. - - yymore() handling rewritten to fix bug with interaction - between yymore() and trailing context. - - EOF in actions now generates an error message. - - Bug involving -CFe and generating equivalence classes fixed. - - Bug which made -CF be treated as -Cf fixed. - - Support for SysV tmpnam() added. - - Unused #define's for scanner no longer generated. - - Error messages which are associated with a particular input - line are now all identified with their input line in standard - format. - - % directives which are valid to lex but not to flex are - now ignored instead of generating warnings. - - -DSYS_V flag can now also be specified -DUSG for System V - compilation. - - -Changes between 2.1 beta-test release of June '89 and previous release: - - User-visible: - - - -p flag generates a performance report to stderr. The report - consists of comments regarding features of the scanner rules - which result in slower scanners. - - - -b flag generates backtracking information to lex.backtrack. - This is a list of scanner states which require backtracking - and the characters on which they do so. By adding rules - one can remove backtracking states. If all backtracking states - are eliminated, the generated scanner will run faster. - Backtracking is not yet documented in the manual entry. - - - Variable trailing context now works, i.e., one can have - rules like "(foo)*/[ \t]*bletch". Some trailing context - patterns still cannot be properly matched and generate - error messages. These are patterns where the ending of the - first part of the rule matches the beginning of the second - part, such as "zx*/xy*", where the 'x*' matches the 'x' at - the beginning of the trailing context. Lex won't get these - patterns right either. - - - Faster scanners. - - - End-of-file rules. The special rule "<<EOF>>" indicates - actions which are to be taken when an end-of-file is - encountered and yywrap() returns non-zero (i.e., indicates - no further files to process). See manual entry for example. - - - The -r (reject used) flag is gone. flex now scans the input - for occurrences of the string "REJECT" to determine if the - action is needed. It tries to be intelligent about this but - can be fooled. One can force the presence or absence of - REJECT by adding a line in the first section of the form - "%used REJECT" or "%unused REJECT". - - - yymore() has been implemented. Similarly to REJECT, flex - detects the use of yymore(), which can be overridden using - "%used" or "%unused". - - - Patterns like "x{0,3}" now work (i.e., with lower-limit == 0). - - - Removed '\^x' for ctrl-x misfeature. - - - Added '\a' and '\v' escape sequences. - - - \<digits> now works for octal escape sequences; previously - \0<digits> was required. - - - Better error reporting; line numbers are associated with rules. - - - yyleng is a macro; it cannot be accessed outside of the - scanner source file. - - - yytext and yyleng should not be modified within a flex action. - - - Generated scanners #define the name FLEX_SCANNER. - - - Rules are internally separated by YY_BREAK in lex.yy.c rather - than break, to allow redefinition. - - - The macro YY_USER_ACTION can be redefined to provide an action - which is always executed prior to the matched rule's action. - - - yyrestart() is a new action which can be used to restart - the scanner after it has seen an end-of-file (a "real" one, - that is, one for which yywrap() returned non-zero). It takes - a FILE* argument indicating a new file to scan and sets - things up so that a subsequent call to yylex() will start - scanning that file. - - - Internal scanner names all preceded by "yy_" - - - lex.yy.c is deleted if errors are encountered during processing. - - - Comments may be put in the first section of the input by preceding - them with '#'. - - - - Other changes: - - - Some portability-related bugs fixed, in particular for machines - with unsigned characters or sizeof( int* ) != sizeof( int ). - Also, tweaks for VMS and Microsoft C (MS-DOS), and identifiers all - trimmed to be 31 or fewer characters. Shortened file names - for dinosaur OS's. Checks for allocating > 64K memory - on 16 bit'ers. Amiga tweaks. Compiles using gcc on a Sun-3. - - Compressed and fast scanner skeletons merged. - - Skeleton header files done away with. - - Generated scanner uses prototypes and "const" for __STDC__. - - -DSV flag is now -DSYS_V for System V compilation. - - Removed all references to FTL language. - - Software now covered by BSD Copyright. - - flex will replace lex in subsequent BSD releases. |