Page 1 of 2
script command gettoken should obey its separator
Posted: 30 Mar 2012 20:41
by neutrox
if I got the concept right this sample script should not pop the "wrong! false string detected" notice.
the problem: if a string has no separator at all, gettoken should not return anything as the "prsence of a separator" condition is false!
in other words, gettoken should only work if its explicit separator is present, returning empty
Code: Select all
$separator = " // ";
$string = "some stuff";
$get = gettoken($string, 1, $separator);
echo "token: $get";
if ($get == "") { echo "right! string not found"; }
elseif ($get != "") { echo "wrong! false string detected"; }
Re: script command gettoken should obey its separator
Posted: 30 Mar 2012 22:06
by highend
Imho it's correct how gettoken works.
Everything _until_ the first separator is found, is the first token. If no separator is found, the full string is the first (and only) token.
Just check if the separator is present via strpos() if you don't like the current behavior.
Re: script command gettoken should obey its separator
Posted: 30 Mar 2012 23:51
by PeterH
highend wrote:Imho it's correct how gettoken works.
Everything _until_ the first separator is found, is the first token. If no separator is found, the full string is the first (and only) token.
Just check if the separator is present via strpos() if you don't like the current behavior.
I'm fully with this statement!
The separator separates multiple tokens - if there is no separator there is just 1 token.
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 02:52
by neutrox
sorry guys but i have to disagree.
a separator acts as a marker, if there's nothing to be marked, then nothing should be returned.
using a gettoken(string, X, separator) only makes sense when a separator is expected and if there's nothing on it's index position, then nothing is actually there.
in the other hand, why use gettoken simply to get an unmarked string?
current behavior is breaking my script and now i'm afraid this is potentially breaking other scripts too. i'll check on highend's suggestion for the situation i'm in but how to check the other stuff flying all around?
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 09:12
by highend
if there's nothing to be marked, then nothing should be returned
Your logic means: There is a string. It is not separated. Great, treat it like it doesn't exist.
Out logic is: If there is no separator, return the full string, because
this is the FIRST token.
why use gettoken simply to get an unmarked string?
You don't have to if you use strpos() before.
current behavior is breaking my script
Maybe you should show the related parts in that case?
i'm afraid this is potentially breaking other scripts too.
Haven't seen any problems so far and I use quite a few scripts with gettoken().
but how to check the other stuff flying all around?
A bit too general, elaborate any "other stuff".
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 09:49
by PeterH
Sorry: the parameter isn't named marker, but separator. So it separates multiple tokens.
Once upon a time it was called delimiter (don't remember on which function) and appeared behind each token - even the last. In this case even a single token was followed by the delimiter. But this wouldn't help you either, I think.
Now it's so: if you have 3 tokens you have 2 delimiters, 2 tokens have 1 delimiter, and 1 token has ...
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 17:58
by neutrox
highend wrote:
current behavior is breaking my script
a Maybe you should show the related parts in that case?
i'm afraid this is potentially breaking other scripts too.
b Haven't seen any problems so far and I use quite a few scripts with gettoken().
but how to check the other stuff flying all around?
c A bit too general, elaborate any "other stuff".
a:
Code: Select all
end(getinfo("CountSelected") < 1), "A file is required!";
$sep = "<crlf>";
$base = get("selecteditemspathnames", "$sep");;
$array = "";
$count = gettoken($base, count, "$sep");
foreach($tk, $base, "$sep") {
$tk2 = readfile("$tk");
$link = gettoken($tk2, 1, " <8marker8> ");
if ($link == "") { } elseif ($link != "") { $array = "$array" . "$link$sep"; }
}
if ($array == "") { echo "None was found, sorry!"; } elseif ($array != "") { $array = formatlist($array, e, $sep); $linkscount = gettoken($array, count, "$sep"); writefile("Links.txt", "$array", r); text "8marker8<crlf>Files: $count<crlf>Links: $linkscount<crlf 2>$array"; }
b: me neither until I went into this situation which worries me a lot. yes all i did all of this time was thinking on my way of understanding
c: refer to b, please.
the problem: i have those files i can scan after markers (aka separators) and should get returns only if a marker is found as i must generate indexes of what the marker is intended for.
then i received among the results the whole file content when a file was found having no marker and hell was unleashed
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 18:07
by highend
Supply such a file that should be read, that creates your "error".
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 18:25
by neutrox
i'm getting "extension not allowed" errors... anyway they are simple text whose contents are like this:
Code: Select all
D:\DVD\DVD Settings.txt <8marker8> 20 min of movie: 1/3 CD (233 MB)
10 : 150 MB
608 MB/.VOB: 1/3 CD (21 min)
518 MB/.VOB: 10min21sec
124 MB/.VOB: 4min20sec
109 MB/.VOB: 3min51sec (70 MB/setting)
(file name: DVD_Settings.txt)
problem happens when i get files which do not contain such marker among selection.
the step further is: i need the list so i can backup only those files relevant to a certain task and they were identified by those
<something> separators (markers)
if i get anything other than
expected file paths or whatever is (
or not) at position 1 all system is lost
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 18:42
by highend
Try this:
Code: Select all
end(getinfo("CountSelected") < 1), "A file is required!";
$sep = "<crlf>";
$base = get("selecteditemspathnames", "$sep");
$array = "";
$count = gettoken($base, count, "$sep");
foreach($tk, $base, "$sep") {
$tk2 = readfile("$tk");
if (strpos($tk2, " <8marker8> ") != -1) {
$link = gettoken($tk2, 1, " <8marker8> ");
if ($link != "") { $array = "$array" . "$link$sep"; }
}
}
if ($array == "") { echo "None was found, sorry!"; } elseif ($array != "") { $array = formatlist($array, e, $sep); $linkscount = gettoken($array, count, "$sep"); writefile("Links.txt", "$array", r); text "8marker8<crlf>Files: $count<crlf>Links: $linkscount<crlf 2>$array"; }
Ofc it doesn't process files, that don't contain the separator.
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 19:11
by neutrox
it works as you suggested in your first reply, highend.
it's the logic that still annoys me...
not to mention i will have to scan and rewrite several scripts, some already on the hands of friends and work colleagues.
sigh.
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 20:20
by Filehero
neutrox wrote:it's the logic that still annoys me...
But that "annoying" logic has been the established view for decades:
If, for the lack of token delimiters, the input cannot be tokenized (split) leave it untouched (unsplit) and simply return it.
Cheers,
Filehero
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 21:13
by neutrox
when I put a fence into my property I want to know what's inside of it, not the whole world beyond it in case the fence falls. If "nothing" is inside the fence, then "nothing" is what I have. To know the whole world beyond I have other tools to count on, same goes for obtaining a simple string in the scope of a script.
If I state a separator and it is not found I must be notified that nothing meets the explicit condition, ie no separator found = nothing there for me to see otherwise I should have used another condition.
Filehero's argument only means to me that no user has come into trouble so far. If I found a problem here i am posting it for discussion. But automatic approval if no discussion has been made so far? Come on, not even the church does that way.
i am talking about a real problem here. Could the objectors kindly tell me which way such change if implemented could affect them, please?
hey, xyplorer's scripting can be exciting too? i should've discovered this years ago....
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 21:27
by highend
Filehero's argument only means to me that no user has come into trouble so far.
Your argument means: You're standing on a your property with a fence around it. When the fence falls, the ground you're standing on is still your property and not "nothing".
The string is still a string. If it isn't separated, the full string is the first token. If it would be empty, the first token would be empty as well. It's a simple logic and I don't see any need to change this. It's intended.
eod (at least from my side)
Re: script command gettoken should obey its separator
Posted: 31 Mar 2012 21:35
by Filehero
Hi neutrox,
neutrox wrote:Filehero's argument only means to me that no user has come into trouble so far.
no, that means the misconception about the
tokenizing paradigm is on your side.
I think, it is not about who's right and wrong. It cannot. We can easily discuss "half full / half empty" until all ends.
But
Tokenizing has alyways worked that way:
highend wrote:Everything _until_ the first separator is found, is the first token. If no separator is found, the full string is the first (and only) token.
Cheers,
Filehero