View unanswered posts | View active topics It is currently Tue Apr 21, 2026 7:41 am



Reply to topic  [ 34 posts ]  Go to page 1, 2, 3  Next
 A SWATH vs. TWX challenge 
Author Message
Commander
User avatar

Joined: Mon Oct 29, 2001 3:00 am
Posts: 1096
Location: Tucson, AZ
Unread post A SWATH vs. TWX challenge
A lot of people say the SWATH parser is slow. I don't believe it. I think it's more that the SWATH display is slow. And the display can be turned off and on by scripts.

But here's an opportunity for the TWX faithful to prove me wrong.

I've written a FastZTM script that uses an optimized version of the "distance map" algorithm from my 2003 ZTM script. The distance map is intended for optimizing probe exploration. It plots from a given base sector (the default being Stardock) to every other sector. In my new version, it looks at the courses already plotted, and if a sector has been seen on the way to a more distant sector, it doesn't plot that course again. It's able to skip about 42% of the universe that way. (Oh, and it checks which sectors have been seen and prepares the next CIM burst in another thread, while the current CIM burst is being parsed in the main thread. Put that in your TWX and smoke it!)

My challenge is for someone to write a TWX script that uses my distance map algorithm... optimized however you like. You're the master TWX scripter, not me! I'll put your script head to head against mine on my local TWGS and report the results here.

_________________
Suddenly you're Busted!


Fri Aug 19, 2011 11:54 pm
Profile WWW
Ambassador
User avatar

Joined: Mon Feb 09, 2004 3:00 am
Posts: 3141
Location: Kansas
Unread post Re: A SWATH vs. TWX challenge
Sounds very similar to what I (as well as SupG, Cherokee, and I think Rev) do on the first pass - skipping found sectors. I don't know the performance under the new TWGS, but performance under the non-beta TWGS can be found at:

viewtopic.php?f=15&t=21264&hilit=proztm


The TWX source to version 5 is at:

viewtopic.php?f=15&t=25325&p=195430&hilit=proztm#p195430 (source)

_________________
               / Promethius / Enigma / Wolfen /

"A man who has no skills can be taught, a man who has no honor has nothing."


Sat Aug 20, 2011 12:18 am
Profile ICQ
Commander
User avatar

Joined: Mon Oct 29, 2001 3:00 am
Posts: 1096
Location: Tucson, AZ
Unread post Re: A SWATH vs. TWX challenge
Is your Pass 4 similar to what I've described?

_________________
Suddenly you're Busted!


Sat Aug 20, 2011 1:02 am
Profile WWW
Ambassador
User avatar

Joined: Mon Feb 09, 2004 3:00 am
Posts: 3141
Location: Kansas
Unread post Re: A SWATH vs. TWX challenge
Mongoose wrote:
Is your Pass 4 similar to what I've described?


Pass 1 is similar but a major difference is it does not use a static sector number to plot from. If I remember correctly it does a double plot say:
123 -> 321, 321 ->123 with the plots switching to ploting a single plot line when the results of the back plot fall below a certain point.

I think I said the source to 5.0 is online, but it is 4.11 I think and it is somewhat different than the thread that has results.

I have heard that the speed of the new TWGS plotting is extremely quick, but I haven't tried it to see.

_________________
               / Promethius / Enigma / Wolfen /

"A man who has no skills can be taught, a man who has no honor has nothing."


Sat Aug 20, 2011 1:21 am
Profile ICQ
Commander
User avatar

Joined: Mon Oct 29, 2001 3:00 am
Posts: 1096
Location: Tucson, AZ
Unread post Re: A SWATH vs. TWX challenge
So it takes an hour and fourteen minutes to plot somewhere between 20,000 and 40,000 routes, depending on how many get dropped by the optimizer? Am I reading that right?

We've got to be comparing apples and oranges, because I'm plotting about 6,000 routes (optimized down from 10,000) in a little over five minutes. (Keep in mind, this is a local game.)

_________________
Suddenly you're Busted!


Sat Aug 20, 2011 1:59 am
Profile WWW
Ambassador
User avatar

Joined: Mon Feb 09, 2004 3:00 am
Posts: 3141
Location: Kansas
Unread post Re: A SWATH vs. TWX challenge
Mongoose wrote:
So it takes an hour and fourteen minutes to plot somewhere between 20,000 and 40,000 routes, depending on how many get dropped by the optimizer? Am I reading that right?

We've got to be comparing apples and oranges, because I'm plotting about 6,000 routes (optimized down from 10,000) in a little over five minutes. (Keep in mind, this is a local game.)


Possibly are. This is not from the current TWGS which has changed the plot speed from what I understand. My results are from the previous TWGS that seemed to pause every so often. The script also has a pause built in to gather information to display on its progress, number of warps found, 7-warps in, and efficiency. I can setup a 30k bang and see what it does on the new version.

_________________
               / Promethius / Enigma / Wolfen /

"A man who has no skills can be taught, a man who has no honor has nothing."


Sat Aug 20, 2011 2:27 am
Profile ICQ
Ambassador
User avatar

Joined: Mon Feb 09, 2004 3:00 am
Posts: 3141
Location: Kansas
Unread post Re: A SWATH vs. TWX challenge
I ran a quick ztm on a 20k under the new TWGS on my local server.. The plot average on pass 1 was 6 warps per plot so it ran about 7,800 plots in almost 13 minutes. The script pauses at every 1% completion to update the pass 1 line for approximately 2 seconds. I give up speed there to feed my desire for what the script is doing, and it is also where the "who's online" check is made.

Attachment:
ztm.jpg
ztm.jpg [ 24.17 KiB | Viewed 9425 times ]

_________________
               / Promethius / Enigma / Wolfen /

"A man who has no skills can be taught, a man who has no honor has nothing."


Sat Aug 20, 2011 3:12 am
Profile ICQ
Commander
User avatar

Joined: Fri Jun 09, 2006 2:00 am
Posts: 1401
Location: Canada
Unread post Re: A SWATH vs. TWX challenge
2 Questions:

What "distant map" algorithm are you speaking of (perhaps you should attach it to your post)?

I'm a little unsure about your challenge.. are you strictly plotting courses for probes to run, irrespective of real world game conditions --ie: blocked sectors, etc. ?

If this is the case I believe a retired player (Traitor), already wrote a script that does that somewhat efficiently (I think I have a copy kicking around somewhere). If I'm not mistaken he started out by simply plotting furthest DE's, and then it's a simple continuation of furthest sector after that. Nothing to it really. His script actually sent out probes, so I have no idea how real efficient it can be.


Image


Attachments:
File comment: ZTM Script. Also Creates a Traffic report.
LSZTM_11.ts [27.19 KiB]
Downloaded 511 times

_________________
----------------------------
-= QUANTUM Computing 101: 15 = 3 x 5 ... 48% of the time.
Sat Aug 20, 2011 6:58 am
Profile ICQ YIM
Veteran Op
User avatar

Joined: Thu Jun 02, 2005 2:00 am
Posts: 5558
Location: USA
Unread post Re: A SWATH vs. TWX challenge
ZTM is an accuracy for time trade-off. The more accurate you want to be, the more time you have to spend doing it. This has nothing to do with Swath or it's speed (especially given plot delay). If you really want to test the processing speed difference, use worldSST w/o a rob delay.

I encourage you to use the little search feature at the top. Most of this has been done before.

_________________
May the unholy fires of corbomite ignite deep within the depths of your soul...

1. TWGS server @ twgs.navhaz.com
2. The NavHaz Junction - Tradewars 2002 Scripts, Resources and Downloads
3. Open IRC chat @ irc.freenode.net:6667 #twchan
4. Parrothead wrote: Jesus wouldn't Subspace Crawl.

*** SG memorial donations via paypal to: dpocky68@booinc.com
Image


Sat Aug 20, 2011 7:44 am
Profile ICQ WWW
Commander
User avatar

Joined: Tue Oct 07, 2003 2:00 am
Posts: 1134
Location: Augusta, GA
Unread post Re: A SWATH vs. TWX challenge
[I've edited this post and script to make it queue 3 plots against the server. I mention this below as well. -EP]

This is an older ZTM technique, and it has been improved upon dramatically. But that isn't really what this challenge is about. You are asking for a TWX script that mirrors your SWATH script. You want it to plot from STARDOCK to every sector in the game, but skip any if they've been seen. This is not a good test of helper or script speed, as the helper's latency is easily hidden by TW's plot delay. But here is a quick TWX example:

Code:
getWord CURRENTLINE $prompt 1
if ($prompt <> "Command")
   echo "*Script must start from the Command prompt."
   halt
end

# Make sure TWX knows where StarDock is
send "v^"
waitOn "StarDock"

setVar $started TIME
setVar $plotCount 0
setVar $wait1 ""
setVar $wait2 ""
setVar $wait3 ""
setVar $plotTo 1
while ($plotTo <= SECTORS)
   if (SECTOR.EXPLORED[$plotTo] = "NO") and ($plotTo <> STARDOCK)
      add $plotCount 1
      send "f" STARDOCK "*" $plotTo "*"
      if ($wait1 <> "")
         waitOn "TO > " & $wait1
         waitOn $wait1
      end
      setVar $wait1 $wait2
      setVar $wait2 $wait3
      setVar $wait3 $plotTo
   end
   add $plotTo 1
end
send "q"
waitOn "Command"
echo "*Plot Count: " $plotCount "*Started: " $started ", Ended: " TIME


Save that as a .ts file, then start it with a clean TWX database. It will echo the plot count at the end, which should be roughly equivalent to the number of plots an apples-to-apples SWATH script takes. Locally, that completes a 10K universe in 3mins, 48secs. Regardless, you have to appreciate how simple that is to do in TWX.

I've optimized ZTM scripts a lot, but always based on the 500ms plot delay previously imposed. Some testing shows that locally at least, queuing 3 plots against the server yields the fastest time, but might result in a few more plots.

_________________
Claim to Fame: only guy to ever crack the TW haggle algorithm, and fig/shield/hold price formula, twice.


Sat Aug 20, 2011 9:44 am
Profile WWW
Commander
User avatar

Joined: Fri Jun 09, 2006 2:00 am
Posts: 1401
Location: Canada
Unread post Re: A SWATH vs. TWX challenge
Okay (If I'm understanding this challenge correctly). here's my submission.. it depends upon a complete ZTM...

Code:
setarray $PLOTTED    SECTORS
setarray $PLOTS    500
setvar $DEBUG       FALSE

setvar $PLOTS_MADE 0
setvar $PLOTS_PARSED 0
setvar $PLOTS_TOTAL 0

Echo ANSI_15 & "**Standby: Generating Target List...**"
setvar $i 1
while ($i <= SECTORS)
   if ($i <> STARDOCK)
      getdistance $DIST STARDOCK $i
      if ($DIST <> "-1")
         if ($PLOTS[$DIST] = "0")
            setvar $PLOTS[$DIST] ""
         end
         setvar $PLOTS[$DIST] ($PLOTS[$DIST] & " " & $i & " ")
      end
   end
   add $i 1
end

if ($DEBUG)
   setvar $FILE "C:\LS_123abc.txt"
   Echo "*Writing Results to : " & $FILE & "*"
   delete $FILE
   setvar $i 1
   while ($i <= 500)
      write $FILE $PLOTS[$i]
      add $i 1
   end
end

Echo "**Plotting...**"
setvar $i 500
while ($i >= 1)
   if ($PLOTS[$i] <> "0")
      getWord $PLOTS[$i] $TEMP 1 "!!@@##"
      while ($TEMP <> "!!@@##")
         add $PLOTS_TOTAL 1
         ReplaceText $PLOTS[$i] (" " & $TEMP & " ") " "
         if ($PLOTTED[$TEMP] <> TRUE)
            add $PLOTS_MADE 1
            getcourse $COURSE STARDOCK $TEMP
             if ($COURSE <> "-1")
               setvar $IDX 1
               while ($IDX <= $COURSE)
                  if ($PLOTTED[$COURSE[$IDX]] <> TRUE)
                     add $PLOTS_PARSED 1
                  end
                  setvar $PLOTTED[$COURSE[$IDX]] TRUE
                  add $IDX 1
               end
            end
         end
         getWord $PLOTS[$i] $TEMP 1 "!!@@##"
      end
      gosub :SLEEP
   end
   subtract $i 1
end

Echo "**" & ANSI_15 & "-----------Results------------"
Echo "*" & ANSI_15 & "Plots Made     : " & $PLOTS_MADE
Echo "*" & ANSI_15 & "Plots Parsed   : " & $PLOTS_PARSED
Echo "**"& ANSI_15 & "Targets Scanned: " & $PLOTS_TOTAL
Echo "**"
Halt

:SLEEP
   setdelaytrigger SLEE   :WAKEUP 10
   pause
   :WAKEUP
   killalltriggers
   return


Couple things. I added the $DEBUG, so you can see the results of the target list (just set it to TRUE); and I added the :SLEEP routine so that I can break out of the cycle without having to End-Task TWX. So, for absolute best speed, just commenet out the Gosub :SLEEP, and make sure $DEBUG is FALSE.

Let me know if Smoke comes out of your System Unit 8)


Attachments:
File comment: As above, so shall it be below
plot.ts [1.62 KiB]
Downloaded 531 times

_________________
----------------------------
-= QUANTUM Computing 101: 15 = 3 x 5 ... 48% of the time.
Sat Aug 20, 2011 11:26 am
Profile ICQ YIM
Commander
User avatar

Joined: Mon Oct 29, 2001 3:00 am
Posts: 1096
Location: Tucson, AZ
Unread post Re: A SWATH vs. TWX challenge
All right. I ran EP's TWX script and my SWATH script on the same local game. Long story short: TWX is faster by about 8%.

EP correctly understood my challenge, and his algorithm is basically the same as mine. TWX definitely has an advantage in simplicity, which makes it easier to optimize. I discovered in my experiments that bursting out 100 plots at a time only results in about 1% more plots than queueing a smaller number, so that's what my script does. (The actual difference in this test was 0.85%, 5665 plots for EP's algo vs. 5713 for mine.)

The total time for a 10K sector universe was 235 seconds for EP's script and 258 seconds for mine. I adjusted this by the number of plots to come up with the 8% speed difference I mentioned above. I suspect that when you figure in network latency, the difference in performance would become even smaller, percentage wise.

Conclusions:
- The TWX version is waaaaaay simpler to write and understand.
- The SWATH version has a pretty GUI form where you input your parameters, and a ANSI progress bar. :lol:
- SWATH's Achilles heel is not its parser, script engine, or database, but its horrendously slow display. Only by turning off the display can performance similar to TWX be achieved.

I ended up ditching the multi-threaded version. Sometimes it worked well, but other times it flaked out or even crashed SWATH. The zip contains the source of my single-threaded FastZTM script, which is unfinished (only the distance map algo has been written) and some utility classes.


Attachments:
FastZTM.zip [4.06 KiB]
Downloaded 529 times

_________________
Suddenly you're Busted!
Sat Aug 20, 2011 6:57 pm
Profile WWW
Gameop
User avatar

Joined: Mon Apr 16, 2001 2:00 am
Posts: 419
Location: Denver Colorado
Unread post Re: A SWATH vs. TWX challenge
LOL, don't any of you guys have girl friends?

_________________
twarbase.com:23


Sat Aug 20, 2011 7:27 pm
Profile ICQ WWW
Ambassador
User avatar

Joined: Mon Feb 09, 2004 3:00 am
Posts: 3141
Location: Kansas
Unread post Re: A SWATH vs. TWX challenge
Master Blaster wrote:
LOL, don't any of you guys have girl friends?


laff, just a wife I've been married to probably longer than many of you have been alive. ;)

_________________
               / Promethius / Enigma / Wolfen /

"A man who has no skills can be taught, a man who has no honor has nothing."


Sat Aug 20, 2011 10:53 pm
Profile ICQ
Commander
User avatar

Joined: Tue Oct 07, 2003 2:00 am
Posts: 1134
Location: Augusta, GA
Unread post Re: A SWATH vs. TWX challenge
Wahoo you made my day, I may have actually converted a programmer from Java/SWATH to TWX!

Boohoo, MB ruined my day, but then he was trolling and posting too, so what's that say about him :P

I do appreciate the candid feedback. In another thread I said TWX is like a domain-specific language. You can say it's inelegant, you can say it's like BASIC, you can say it's not extensible. It has these and other flaws. But for scripting common TW tasks, which is what it's all about, you have to concede that it's fast, simple, and easily understood... assuming the script is well written of course.

_________________
Claim to Fame: only guy to ever crack the TW haggle algorithm, and fig/shield/hold price formula, twice.


Sat Aug 20, 2011 11:53 pm
Profile WWW
Display posts from previous:  Sort by  
Reply to topic   [ 34 posts ]  Go to page 1, 2, 3  Next

Who is online

Users browsing this forum: No registered users and 22 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by wSTSoftware.