Developer guide

API

Producers

poultry.producers.consume_stream(target, input_dir=None)

Read lines from the standard input or files in a directory.

Behaves as a generator, should receive a .send() call to send a line to the target.

poultry.producers.from_stream(target, source=None, config=None, extract_retweets=False)

Send lines from the standard input, the input directory or the Twitter Streaming API.

Parameters:

• target – a generator to which the read lines are sent.
• source – the path to a directory with tweet files.
• config – the config file
• extract_retweets – Extract retweets from the tweets.

poultry.producers.readline_dir(input_dir, extract_retweets=False, mark_extracted=False)

Read a tweet collection directory.

Parameters:input_dir (str) – the patht to the directory Returns:an iterable of Tweet objects

Consumers

exception poultry.consumers.BatchEndException(last_item, batch_size)

Is thrown by batch() to the target generator on the end of the current batch.

class poultry.consumers.SendNext

Returned a consumer if the sent value is ignored, and the next value should be sent immediately.

poultry.consumers.closing(*args, **kwds)

Throw exceptions to targets and close targets on exit.

poultry.consumers.consumer(func)

A decorator function that takes care of starting a coroutine automatically on call.

See http://www.dabeaz.com/generators/ for more details.

poultry.consumers.count_tokens(*counters, **kwargs)

Count tokens in a tweet.

poultry.consumers.lazy_counter(*args, **kwds)

Delay the update of the counter.

poultry.consumers.timeline(*counters, **kwargs)

Count tweet’s creation time.

Tweet

class poultry.tweet.Coordinates

exception poultry.tweet.TweetValueError

Thrown when a class:Tweet can’t be built.

Stream

exception poultry.stream.EndOfStreamError

Twitter streams are supposed to be infinite.

The exception should be thrown, if for any reason the stream has ended.

class poultry.stream.StreamConsumer(queue, target, *args, **kwargs)

A consumer of a stream of tweets.

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

class poultry.stream.StreamProducer(target, twitter_credentials, follow=None, track=None, locations=None, language=None, url='https://stream.twitter.com/1.1/statuses/filter.json', *args, **kwargs)

A producer of a tweet stream retrieved from twitter.

Parameters:

• target – A coroutine to which collected tweets are sent.
• twitter_credentials – A dictionary with access_token,

access_token_secret, consumer_key and consumer_secret. :param follow: A list of user ids to follow. :param track: A list of phrases to track. :param locations: A list of coordinates to get.

..todo:: Parameter description!

The default access level allows up to 400 track keywords, 5,000 follow userids and 25 0.1-360 degree location boxes. https://dev.twitter.com/docs/streaming-api/methods

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

poultry.stream.from_twitter_api(target, endpoint, config)

Consume tweets from a Streaming API endpoint.