Just when I was about to give up I came across a tool that not only does what it says but also does so much more. Sitemap Generator from WonderWebWare was just what I was looking for. Below is how to run this and other tools from WonderWebWare on Linux, Mac, FreeBSD, and OpenSolaris with Bordeaux or vanilla Wine from WineHQ.
I ran the programs on Linux but they should also run on any platform that Wine runs on.
Sitemap Generator is a free software that will help you to automatically generate sitemaps in google (XML) or yahoo (TXT) format or just to spider your website and collect all the internal links for you. The tool may export collected links to CSV or HTML file too. The main reason to build this tool was that all other similar programs didnt fit my requirements.
This software is free for any use, but comes without any warranty. Please, note that the program acts like a spider, but will download all the pages of the target site. Do not use this tool to crawl 3-rd party sites without their permission.
How it works:
Although the program isnt very simple, the way it generates the map is simple, it acts like a web-spider. When you give the starting url to the tool, it will start crawling your site, extracting all the links from all the pages.
Download and setup on Linux with Bordeaux or WineSiteMap Generator Download
Bordeaux version used 2.0.4 for Linux
Wine 1.1.44 or higher should also work
This howto is based on installing and using SiteMap Generator with Bordeaux but a recent version of Wine from WineHQ should also work without any problems.
After you download SiteMap Generator you will need to install it as a unsupported application with Bordeaux. To do this is extremely simple just run the installer with Bordeaux bundled Wine from the command line and install the software.
$ /opt/bordeaux/bin/wine SiteMapG.exe
The program should install without any errors, after the install has completed you will need to cd to the install directory and then run SiteMapGen.exe
$ cd /home/tom/.wine/drive_c/Program\ Files/SiteMap\ Generator
$ /opt/bordeaux/bin/wine SiteMapGen.exe
At this point SiteMap Generator should run without any problems.
Below is some screenshots of SiteMap Genorator running on my laptop with Ubuntu Linux installed. I created sitemaps for http://www.wine-reviews.net/ and http://www.bordeauxgroup.com/ and submitted them to Google and Yahoo!
I also wanted to use "Site Submit" to Submit my sites to about 5000 seo friendly ("dofollow") directories.
OverviewSite Submit is the new kid in the town: a simple semi-automated website submission tool which can speed up the directory submission process. The main advantage is that it's free, however (if the public is interested) I plan to add more features, more directories and some sort of online updates... The tool comes with about 5000 seo-friendly ("dofollow") directories in the database.
Site Submit also works near perfect if you install it into a IE 6 or 7 Cellar. This is verry easy to do because at this time Site Submit comes in a zip archive. After you install IE with Bordeaux or Wine all you need to do is un zip Site Submit in the Programs Directory and then cd to the SiteSubmit directory and then run SiteSubmit.exe
$ cd /home/tom/.bordeaux/ie6/drive_c/Program\ Files/SiteSubmit
$ /opt/bordeaux/bin/wine SiteSubmit.exe