In my earlier publish, I talked about learn how to mix the Java library Twitter4J and Stata’s Java operate Interface utilizing Eclipse to create a helloWorld plugin. Now, I need to speak about learn how to name Twitter4j member features to hook up with Twitter REST API, return Twitter information, and cargo that information into Stata utilizing the Stata SFI.
Including twitter4J embrace information and globl
The present code is
bundle com.stata.kcrow;
import com.stata.sfi.*;
public class StTwitter {
public static int HelloWorld(String args[]) {
SFIToolkit.error("Hey World!");
return(0);
}
}
To make use of the twitter4J operate calls, I want so as to add the next code to the highest of our StTwitter.java file:
import twitter4j.*; import twitter4j.conf.ConfigurationBuilder;
Additionally, I want so as to add a world Twitter class to our StTwitter class. My code now reads
bundle com.stata.kcrow;
import com.stata.sfi.*;
import twitter4j.*;
import twitter4j.conf.ConfigurationBuilder;
public class StTwitter {
static Twitter twitter;
public static int HelloWorld(String args[]) {
SFIToolkit.error("Hey World!");
return(0);
}
}
Occasion member operate
Most web site APIs use some type of OAuth for authentication with their servers. Twitter makes use of OAuth2. twitter2stata makes use of Twitter’s application-based authentication mannequin, however there are different methods to attach. For this publish, I may also use the application-based authentication mannequin.
The primary operate I want to put in writing is a operate to authenticate to the Twitter web site API. To do that, you want to get your Client Key (API Key), Client Secret (API Secret), Entry Token, and Entry Token Secret strings. In case you don’t bear in mind learn how to get these, see my earlier weblog publish right here.
After you have these tokens, you’ll be able to then write a easy operate to go this data utilizing the category member operate ConfigurationBuilder:
non-public static void getInstance() {
ConfigurationBuilder cb;
TwitterFactory tf;
cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey(CONSUMER_KEY)
.setOAuthConsumerSecret(CONSUMER_SECRET)
.setOAuthAccessToken(ACCESS_TOKEN)
.setOAuthAccessTokenSecret(ACCESS_TOKEN_SECRET);
tf = new TwitterFactory(cb.construct());
twitter = tf.getInstance();
}
This operate creates a ConfigurationBuilder occasion, units the login settings, creates a Twitter manufacturing facility occasion, after which initializes the worldwide Twitter class occasion. My class now reads
bundle com.stata.kcrow;
import com.stata.sfi.*;
import twitter4j.*;
import twitter4j.conf.ConfigurationBuilder;
public class StTwitter {
static Twitter twitter ;
non-public static void getInstance() {
ConfigurationBuilder cb;
TwitterFactory tf;
cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey("xWNlx*N9vESv0ZZBtGdm7fVB")
.setOAuthConsumerSecret("7D25oVzWeDCHrUlQcp9929@GOcnqWCuUKhDel")
.setOAuthAccessToken("74741598400768-3hAYpZbiDvABPizx5lk57B8CTVyfa")
.setOAuthAccessTokenSecret("7HjDf25oVzDWAeDCHrUlQcpfNGOTzcnqWCuUKhDel");
tf = new TwitterFactory(cb.construct());
twitter = tf.getInstance();
}
public static int HelloWorld(String args[]) {
SFIToolkit.error("Hey World!");
return(0);
}
}
I made this class non-public as a result of this operate doesn’t must be referred to as from Stata.
Search member operate
Now that I’ve written our occasion operate, I can write our search operate. The Twitter4J features I want are
I’ll use Question to set our search setttings, search() to fetch the search outcomes, and getRateLimitStatus() to make sure I don’t go over the Twitter fee limits. Let’s write our operate to fetch search outcomes 100 at a time and hit the fee limits. Final, I’ll use getTweets() to fetch the Standing objects (actually tweet object). I can loop the Standing objects returned by search() utilizing a for and use a do/whereas loop to verify I’ve outcomes. I coded
public static int searchTweets(String[] args) {
int rc, restrict;
String search_query;
Question question;
QueryResult end result;
getInstance(); //Name static member operate above
search_query = args[0]; // Argument handed from Stata.
question = new Question(search_query);
question.setCount(100);
end result = null;
do {
end result = twitter.search(question);
restrict = end result.getRateLimitStatus().getRemaining();
for (Standing tweet_object: end result.getTweets()) {
//course of information
}
} whereas (restrict > 0);
return(rc);
}
In Java, it is suggested that you just put code that would lead to an error within the Attempt/Catch block. This makes error dealing with simpler. You should utilize the TwitterException class to assist with error dealing with. This class can generate extra particular error messages, however I cannot use it. For this instance, I’ll concern a generic couldn’t search tweets error for any TwitterException.
public static int searchTweets(String[] args) {
int rc, restrict;
String search_query;
Question question;
QueryResult end result;
getInstance(); //Name static member operate above
search_query = args[0]; // Argument handed from Stata.
question = new Question(search_query);
question.setCount(100);
end result = null;
attempt {
do {
end result = twitter.search(question);
restrict = end result.getRateLimitStatus().getRemaining();
for (Standing tweet_object: end result.getTweets()) {
//course of information
}
} whereas (restrict > 0);
}
catch (TwitterException te) {
SFIToolkit.errorln("couldn't search tweets");
rc = 606
}
return(rc);
}
To date, I’ve used two Stata SFI member features:
Each of those features show an error within the Stata Outcomes home windows. The one distinction between the 2 is that SFIToolkit.errorln provides a line terminator on the finish of the error. Now, let’s take a look at the Stata SFI Information class used to course of the information returned from Twitter.
Write information into Stata
To course of the information from Twitter, I have to first add each variables and observations to Stata. So as to add observations to Stata, you employ the SFI Information class operate stObsTotal(). There are a number of features so as to add variables, relying on the kind.
I’ll use addVarStr() and addVarDouble() for this instance. The tweet information Twitter returns is organized into two sorts of information: the tweet object and the consumer object. For this instance, we’re going to course of just a few of the metadata from each objects. From the tweet object I’ll course of:
- textual content
- retweet_count
- favorite_count
From the consumer object, I’ll course of:
- screen_name
- followers_count
- friends_count
In our searchTweets operate, I want so as to add the next code to create our variables and add observations to Stata:
public static int searchTweets(String[] args) {
int rc, restrict;
String search_query;
Question question;
QueryResult end result;
lengthy obs ;
getInstance(); //Name static member operate above
search_query = args[0]; // Argument handed from Stata.
question = new Question(search_query);
question.setCount(100);
end result = null;
//Create variables
rc = Information.addVarStr("textual content", 240);
if (rc!=0) return(rc);
rc = Information.addVarDouble("retweet_count");
if (rc!=0) return(rc);
rc = Information.addVarDouble("favorite_count");
if (rc!=0) return(rc);
rc = Information.addVarStr("screen_name", 30);
if (rc!=0) return(rc);
rc = Information.addVarDouble("followers_count");
if (rc!=0) return(rc);
rc = Information.addVarDouble("friends_count");
if (rc!=0) return(rc);
obs = 0;
attempt {
do {
end result = twitter.search(question);
restrict = end result.getRateLimitStatus().getRemaining();
for (Standing tweet_object: end result.getTweets()) {
//Add observations
obs++;
rc = Information.setObsTotal(obs);
if (rc!=0) return(rc);
//course of information
}
} whereas (restrict > 0);
}
catch (TwitterException te) {
SFIToolkit.errorln("couldn't search tweets");
rc = 606;
}
return(rc);
}
In our searchTweets member operate, I want to put in writing a non-public member operate that copies the outcomes returned from Twitter’s objects into Stata’s reminiscence. I added the under name contained in the for loop.
...
for (Standing tweet_object: end result.getTweets()) {
//Add observations
obs++;
rc = Information.setObsTotal(obs);
if (rc!=0) return(rc);
//course of information
rc = processData(obs, tweet_object,
tweet_object.getUser());
if (rc!=0) return(rc);
}
...
The operate is
non-public static int processData(lengthy obs, Standing tweet_object, Person user_object){
int rc;
rc = Information.storeStr(1, obs, tweet_object.getText());
if(rc) return(rc);
rc = Information.storeNum(2, obs, tweet_object.getRetweetCount());
if(rc) return(rc);
rc = Information.storeNum(3, obs, tweet_object.getFavoriteCount());
if(rc) return(rc);
rc = Information.storeStr(4, obs, user_object.getName());
if(rc) return(rc);
rc = Information.storeNum(5, obs, user_object.getFollowersCount());
if(rc) return(rc);
rc = Information.storeNum(6, obs, user_object.getFriendsCount());
if(rc) return(rc);
return(rc);
}
Observe that the primary argument of the the Information.retailer* is simply the variable index order within the present datasets in reminiscence. You’ll be able to take a look at every operate name to get the metadata for the tweet object right here and for the consumer object right here.
Last class
Our last code to the StTwitter class is
bundle com.stata.kcrow;
import com.stata.sfi.*;
import twitter4j.*;
import twitter4j.conf.ConfigurationBuilder;
public class StTwitter {
static Twitter twitter;
non-public static void getInstance() {
ConfigurationBuilder cb;
TwitterFactory tf;
cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey("xWNlx*N9vESv0ZZBtGdm7fVB")
.setOAuthConsumerSecret("7D25oVzWeDCHrUlQcp9929@GOcnqWCuUKhDel")
.setOAuthAccessToken("74741598400768-3hAYpZbiDvABPizx5lk57B8CTVyfa")
.setOAuthAccessTokenSecret("7HjDf25oVzDWAeDCHrUlQcpfNGOTzcnqWCuUKhDel");
tf = new TwitterFactory(cb.construct());
twitter = tf.getInstance();
}
public static int searchTweets(String[] args) {
int rc, restrict;
String search_query;
Question question;
QueryResult end result;
lengthy obs;
getInstance(); //Name static member operate above
search_query = args[0]; // Argument handed from Stata.
question = new Question(search_query);
question.setCount(100);
end result = null;
//Create variables
rc = Information.addVarStr("textual content", 240);
if (rc!=0) return(rc);
rc = Information.addVarDouble("retweet_count");
if (rc!=0) return(rc);
rc = Information.addVarDouble("favorite_count");
if (rc!=0) return(rc);
rc = Information.addVarStr("screen_name", 30);
if (rc!=0) return(rc);
rc = Information.addVarDouble("followers_count");
if (rc!=0) return(rc);
rc = Information.addVarDouble("friends_count");
if (rc!=0) return(rc);
obs = 0 ;
attempt {
do {
end result = twitter.search(question);
restrict = end result.getRateLimitStatus().getRemaining();
for (Standing tweet_object: end result.getTweets()) {
//Add observations
obs++;
rc = Information.setObsTotal(obs);
if (rc!=0) return(rc);
//course of information
rc = processData(obs, tweet_object,
tweet_object.getUser());
if (rc!=0) return(rc);
}
} whereas (restrict > 0);
}
catch (TwitterException te) {
if (!SFIToolkit.errorDebug(SFIToolkit.
stackTraceToString(te)+"n")) {
SFIToolkit.errorln("couldn't search tweets");
}
rc = 606 ;
}
return(rc);
}
non-public static int processData(lengthy obs, Standing tweet_object, Person user_object){
int rc;
rc = Information.storeStr(1, obs, tweet_object.getText());
if (rc!=0) return(rc);
rc = Information.storeNum(2, obs, tweet_object.getRetweetCount());
if (rc!=0) return(rc);
rc = Information.storeNum(3, obs, tweet_object.getFavoriteCount());
if (rc!=0) return(rc);
rc = Information.storeStr(4, obs, user_object.getName());
if (rc!=0) return(rc);
rc = Information.storeNum(5, obs, user_object.getFollowersCount());
if (rc!=0) return(rc);
rc = Information.storeNum(6, obs, user_object.getFriendsCount());
if (rc!=0) return(rc);
return(rc);
}
public static int HelloWorld(String args[]) {
SFIToolkit.error("Hey World!");
return(0);
}
}
Bundling and redistributing the JAR file
There are two primary methods to make the StTwitter class work in Stata.
-
Copy the Twitter4J .jar information to someplace alongside your adopath. You’ll be able to then use the jar() choice for javacall to specifiy which information to make use of. For instance,
. which twitter4j-core-4.0.4.jar c:adopersonaltwitter4j-core-4.0.4.jar . javacall com.stata.kcrow.StTwitter searchTweets, args("star wars") /// jars(test_twitter.jar;twitter4j-core-4.0.4.jar)I like to recommend this methodology in case you are growing a Java library to your personal use.
-
Export the venture as a Runnable JAR file
If you’re growing a Java library for redistribution to someplace just like the Statistical Software program Elements archive, chances are you’ll need to mix all .jar information into one .jar file.
Click on the Subsequent button, and kind the
path/filethe place you want to the .jar file saved.
Final, click on the End botton.
Additionally, be sure you have the right software program license kind to re-distribute any library .jar file.
Parsing in Stata
With our StTwitter class coded and correctly positioned, I can now add parsing to the ado program:
program outline twitter_test
model 15
args search_string junk
if ("`junk'" != "") {
show as error "invalid syntax"
exit 198
}
javacall com.stata.kcrow.StTwitter searchTweets, ///
args(`"`search_string'"') ///
jars(test_twitter.jar;twitter4j-core-4.0.4.jar)
finish
Save the above file as twitter_test.ado alongside your adopath and, in Stata, kind
. twitter_test "star wars"
.describe
Comprises information
obs: 18,000
vars: 6
dimension: 5,436,000
-------------------------------------------------------------------------------
storage show worth
variable identify kind format label variable label
-------------------------------------------------------------------------------
textual content str240 %240s
retweet_count double %10.0g
favorite_count double %10.0g
screen_name str30 %30s
followers_count double %10.0g
friends_count double %10.0g
-------------------------------------------------------------------------------
Sorted by:
Observe: Dataset has modified since final saved.
Conclusion
As you’ll be able to see, it didn’t take a lot code to hook up with Twitter’s API, return tweet information, and cargo that information into Stata. Twitter does sale enterprise licenses for his or her information, which haven’t any limits on the quantity of knowledge you’ll be able to obtain. You too can fetch information way back to 2006. There’s a completely different API for this information. The Twitter4J library helps this API as nicely, however the twitter2stata command doesn’t.
