You can edit your /home/<user name>/.fluxbox/init file in a text editor.
Find the line -
session.screen0.tabs.usePixmap: true
change to:
session.screen0.tabs.usePixmap: false
There should also be an entry on the default fluxbox configuration menu which is incorporated into the VSIDO menu; I can't remember how to get to it (I'm on Slackware right now).
That icon is actually on the window tab. By default, window tabs are contained in the titlebar. If you use tabbed windows, you can place the tab on the window sides, bottom etc ...
Your link is the old fluxbox wiki - take a look here -
http://fluxbox-wiki.org/index.php?title=Fluxbox-wikiWelcome to VSIDO.