Automating internet downloads and file unzipping

Andrew J. Dyck wrote a pleasant put up on his weblog on the way to Obtain and unzip information recordsdata from Stata. He writes

Not too long ago, I’ve been utilizing Stata’s -shp2dta- command to transform some shapefiles to stata format, grabbing Lat/Lon information and merging into one other dataset. There have been a number of compressed shapefiles I needed to obtain contained in a listing from the net. I may manually obtain every file and uncompress each however that might be time consuming. Additionally, when the maps are up to date, I’d should do the obtain/uncompress over again. I’ve discovered that the method may be automated from inside Stata through the use of a mixture of -shell- and a few useful terminal instructions. …

You need to learn the remainder of his put up. He goes on to indicate how one can script with Stata to automate shelling out to obtain and unzip a sequence of recordsdata from a web site, and he introduces you to some cool Unix-like utilities for Home windows.

We right here at StataCorp use Stata for duties like this on a regular basis. In actual fact, we now have constructed some instruments into Stata to permit you to do a lot of what Andrew described with out ever having to go away or shell out of Stata.

For instance, Stata can entry recordsdata over the Web. Stata has a copy command. And, as of Stata 11, Stata can immediately zip and unzip recordsdata and directories.

Placing all of these capabilities to make use of, you may accomplish Andrew’s aim by writing code immediately in Stata similar to

copy http://instance.com/download1.zip download1.zip
copy http://instance.com/download2.zip download2.zip
unzipfile download1.zip
unzipfile download2.zip

If there have been numerous recordsdata you wished to obtain and unzip, and so they had been all named in a daily method (say, “download1.zip” by way of “download100.zip”), you could possibly deliver all of them down and unzip them immediately in Stata with a 4 line loop:

forvalues i = 1/100 {
    copy http://instance.com/obtain`i'.zip obtain`i'.zip
    unzipfile obtain`i'.zip
}