script command gettoken should obey its separator

[neutrox]

if I got the concept right this sample script should not pop the "wrong! false string detected" notice.

the problem: if a string has no separator at all, gettoken should not return anything as the "prsence of a separator" condition is false!

in other words, gettoken should only work if its explicit separator is present, returning empty

Code: Select all

   $separator = " // ";
   $string = "some stuff";
   $get = gettoken($string, 1, $separator);
   echo "token: $get";
   if ($get == "") { echo "right! string not found"; }
   elseif ($get != "") { echo "wrong! false string detected"; }

[highend]

Imho it's correct how gettoken works.

Everything _until_ the first separator is found, is the first token. If no separator is found, the full string is the first (and only) token.

Just check if the separator is present via strpos() if you don't like the current behavior.

[PeterH]

Imho it's correct how gettoken works.

Everything _until_ the first separator is found, is the first token. If no separator is found, the full string is the first (and only) token.

Just check if the separator is present via strpos() if you don't like the current behavior.

I'm fully with this statement!

The separator separates multiple tokens - if there is no separator there is just 1 token.

[neutrox]

sorry guys but i have to disagree.

a separator acts as a marker, if there's nothing to be marked, then nothing should be returned.

using a gettoken(string, X, separator) only makes sense when a separator is expected and if there's nothing on it's index position, then nothing is actually there.

in the other hand, why use gettoken simply to get an unmarked string?

current behavior is breaking my script and now i'm afraid this is potentially breaking other scripts too. i'll check on highend's suggestion for the situation i'm in but how to check the other stuff flying all around?

[highend]

if there's nothing to be marked, then nothing should be returned

Your logic means: There is a string. It is not separated. Great, treat it like it doesn't exist.
Out logic is: If there is no separator, return the full string, because
this is the FIRST token.

why use gettoken simply to get an unmarked string?

You don't have to if you use strpos() before.

current behavior is breaking my script

Maybe you should show the related parts in that case?

i'm afraid this is potentially breaking other scripts too.

Haven't seen any problems so far and I use quite a few scripts with gettoken().

but how to check the other stuff flying all around?

A bit too general, elaborate any "other stuff".

[PeterH]

Sorry: the parameter isn't named marker, but separator. So it separates multiple tokens.

Once upon a time it was called delimiter (don't remember on which function) and appeared behind each token - even the last. In this case even a single token was followed by the delimiter. But this wouldn't help you either, I think.

Now it's so: if you have 3 tokens you have 2 delimiters, 2 tokens have 1 delimiter, and 1 token has ...

[neutrox]

current behavior is breaking my script

a Maybe you should show the related parts in that case?

i'm afraid this is potentially breaking other scripts too.

b Haven't seen any problems so far and I use quite a few scripts with gettoken().

but how to check the other stuff flying all around?

c A bit too general, elaborate any "other stuff".

a:

Code: Select all

 end(getinfo("CountSelected") < 1), "A file is required!";
 $sep = "<crlf>";
 $base = get("selecteditemspathnames", "$sep");;
 $array = "";
 $count = gettoken($base, count, "$sep");

 foreach($tk, $base, "$sep") {
 $tk2 = readfile("$tk");
 $link = gettoken($tk2, 1, " <8marker8> ");
 if ($link == "") {  } elseif ($link != "") { $array = "$array" . "$link$sep"; }

                             }

 if ($array == "") { echo "None was found, sorry!"; } elseif ($array != "") { $array = formatlist($array, e, $sep); $linkscount = gettoken($array, count, "$sep"); writefile("Links.txt", "$array", r); text "8marker8<crlf>Files: $count<crlf>Links: $linkscount<crlf 2>$array"; }

b: me neither until I went into this situation which worries me a lot. yes all i did all of this time was thinking on my way of understanding

c: refer to b, please.

the problem: i have those files i can scan after markers (aka separators) and should get returns only if a marker is found as i must generate indexes of what the marker is intended for. then i received among the results the whole file content when a file was found having no marker and hell was unleashed

[highend]

Supply such a file that should be read, that creates your "error".

[neutrox]

i'm getting "extension not allowed" errors... anyway they are simple text whose contents are like this:

Code: Select all

D:\DVD\DVD Settings.txt <8marker8> 20 min of movie:  1/3 CD (233 MB)
10             :  150 MB

608 MB/.VOB:      1/3 CD (21 min)

518 MB/.VOB:      10min21sec

124 MB/.VOB:      4min20sec

109 MB/.VOB:      3min51sec (70 MB/setting)

(file name: DVD_Settings.txt)
problem happens when i get files which do not contain such marker among selection.
the step further is: i need the list so i can backup only those files relevant to a certain task and they were identified by those <something> separators (markers)

if i get anything other than expected file paths or whatever is (or not) at position 1 all system is lost

[highend]

Try this:

Code: Select all

 end(getinfo("CountSelected") < 1), "A file is required!";
 $sep = "<crlf>";
 $base = get("selecteditemspathnames", "$sep");
 $array = "";
 $count = gettoken($base, count, "$sep");

 foreach($tk, $base, "$sep") {
   $tk2 = readfile("$tk");
   if (strpos($tk2, " <8marker8> ") != -1) {
     $link = gettoken($tk2, 1, " <8marker8> ");
     if ($link != "") { $array = "$array" . "$link$sep"; }
   }
 }

 if ($array == "") { echo "None was found, sorry!"; } elseif ($array != "") { $array = formatlist($array, e, $sep); $linkscount = gettoken($array, count, "$sep"); writefile("Links.txt", "$array", r); text "8marker8<crlf>Files: $count<crlf>Links: $linkscount<crlf 2>$array"; }
 

Ofc it doesn't process files, that don't contain the separator.

[neutrox]

it works as you suggested in your first reply, highend.

it's the logic that still annoys me...
not to mention i will have to scan and rewrite several scripts, some already on the hands of friends and work colleagues.
sigh.

[Filehero]

it's the logic that still annoys me...

But that "annoying" logic has been the established view for decades:

If, for the lack of token delimiters, the input cannot be tokenized (split) leave it untouched (unsplit) and simply return it.


Cheers,
Filehero

[neutrox]

when I put a fence into my property I want to know what's inside of it, not the whole world beyond it in case the fence falls. If "nothing" is inside the fence, then "nothing" is what I have. To know the whole world beyond I have other tools to count on, same goes for obtaining a simple string in the scope of a script.

If I state a separator and it is not found I must be notified that nothing meets the explicit condition, ie no separator found = nothing there for me to see otherwise I should have used another condition.

Filehero's argument only means to me that no user has come into trouble so far. If I found a problem here i am posting it for discussion. But automatic approval if no discussion has been made so far? Come on, not even the church does that way.

i am talking about a real problem here. Could the objectors kindly tell me which way such change if implemented could affect them, please?

hey, xyplorer's scripting can be exciting too? i should've discovered this years ago....

[highend]

Filehero's argument only means to me that no user has come into trouble so far.

Your argument means: You're standing on a your property with a fence around it. When the fence falls, the ground you're standing on is still your property and not "nothing".

The string is still a string. If it isn't separated, the full string is the first token. If it would be empty, the first token would be empty as well. It's a simple logic and I don't see any need to change this. It's intended.

eod (at least from my side)

[Filehero]

Hi neutrox,

Filehero's argument only means to me that no user has come into trouble so far.

no, that means the misconception about the tokenizing paradigm is on your side.
I think, it is not about who's right and wrong. It cannot. We can easily discuss "half full / half empty" until all ends.

But Tokenizing has alyways worked that way:

Everything _until_ the first separator is found, is the first token. If no separator is found, the full string is the first (and only) token.

Cheers,
Filehero