Wget

wget multiple files wget -r -l1 -A.mp3 http://aaa.com/directory In the above example, -r and -l1 options together enable 1-level deep recursive retrieval, and -A option specifies lists of file name suffixes to accept during recursive download (.mp3 in this case). ways to wget entire webpage This one works so well, I’ve created an alias for it: wgetMirror='/usr/bin/wget -o wget.log -mkEpnp –wait=9 –user-agent='\''Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'\'' –no-check-certificate' Other options: wget --recursive --no-clobber --page-requisites --html-extension --convert-links --restrict-file-names=windows --domains example.com --no-parent <url> Modify User Agent $ wget -U "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" <url> wget from webdav with authentication wget --http-user=user-id --http-password=password <URL> Reference Download Multiple Files with wget Archiving website with wget Download entire webpage with wget Change the User Agent in wget