[Freeciv-Dev] Re: Split patch (was Re: [RFC PATCH] init_techs)

["Ross W. Wetmore" <rwetmore@xxxxxxxxxxxx>]

At 09:05 AM 01/09/28 +0200, Reinier Post wrote:
>On Fri, Sep 28, 2001 at 12:00:07AM -0400, Ross W. Wetmore wrote:
>> split should treat the buffer it was handed as working memory, return
>> pointers into the parsed string elements, and let the caller deal with
>> ALL memory issues.
>> 
>> It is really the only sensible general purpose solution for something 
>> like this.
>
>Only if you do not have to expand the input while tokenizing it.
>Then you can change whitespace to NUL a la strtok() either on the
>original or on a copy.  I think Freeciv will meets this case
>regardless of the extended command syntax Justin decides on.
>
>-- 
>Reinier

I think the restriction is that you always have to separate tokens by
at least some char sized element in the initial buffer.

If the interface definition of "const char *const toks" means that it
matches undelimited strings of toks, this can be a problem. I'm not
precisely sure whether toks are "elements" or "separators" here. I
sort of expect to see tok being separators { "," " " "\t" "\n" }. If
they are elements, I might be tempted to call the routine parse().

But I believe that apart from "=" there is a whitespace separator rule
in addition to any tok matching that makes things ok even in this case. 
"=" is really just another whitespace character, at least in certain 
special places.

If you were going to translate the token strings after this, then that
is a function that should be done in the calling code, and not the
splitting code, so again you shouldn't need to worry about string
expansion.

It should be trivial to write a wrapper, that split, then did something
like hashed token lookups to return any actual expanded items[] that might 
be needed for a given parsing algorithm if you need this.

Note this is an example where you don't want to strdup() the initial 
split() elements, and may not need to do this for the lookup values 
depending on how the overall algorithm was designed. So extra 
strdup()/free() calls just waste performance cycles here.

Cheers,
RossW
=====