Developer guide

API

Producers

poultry.producers.consume_stream(target, input_dir=None)[source]

Read lines from the standard input or files in a directory.

Behaves as a generator, should receive a .send() call to send a line to the target.

poultry.producers.from_stream(target, source=None, config=None, extract_retweets=False)[source]

Send lines from the standard input, the input directory or the Twitter Streaming API.

Parameters:
  • target – a generator to which the read lines are sent.
  • source – the path to a directory with tweet files.
  • config – the config file
  • extract_retweets – Extract retweets from the tweets.
poultry.producers.readline_dir(input_dir, extract_retweets=False, mark_extracted=False)[source]

Read a tweet collection directory.

Parameters:input_dir (str) – the patht to the directory
Returns:an iterable of Tweet objects

Consumers

exception poultry.consumers.BatchEndException(last_item, batch_size)[source]

Is thrown by batch() to the target generator on the end of the current batch.

class poultry.consumers.SendNext[source]

Returned a consumer if the sent value is ignored, and the next value should be sent immediately.

poultry.consumers.closing(*args, **kwds)[source]

Throw exceptions to targets and close targets on exit.

poultry.consumers.consumer(func)[source]

A decorator function that takes care of starting a coroutine automatically on call.

See http://www.dabeaz.com/generators/ for more details.

poultry.consumers.count_tokens(*counters, **kwargs)[source]

Count tokens in a tweet.

poultry.consumers.lazy_counter(*args, **kwds)[source]

Delay the update of the counter.

poultry.consumers.timeline(*counters, **kwargs)[source]

Count tweet’s creation time.

Tweet

exception poultry.tweet.TweetValueError[source]

Thrown when a class:Tweet can’t be built.

Stream

exception poultry.stream.EndOfStreamError[source]

Twitter streams are supposed to be infinite.

The exception should be thrown, if for any reason the stream has ended.

class poultry.stream.StreamConsumer(queue, target, *args, **kwargs)[source]

A consumer of a stream of tweets.

class poultry.stream.StreamProducer(target, twitter_credentials, follow=None, track=None, locations=None, language=None, url='https://stream.twitter.com/1.1/statuses/filter.json', *args, **kwargs)[source]

A producer of a tweet stream retrieved from twitter.

Parameters:
  • target – A coroutine to which collected tweets are sent.
  • twitter_credentials – A dictionary with access_token,

access_token_secret, consumer_key and consumer_secret. :param follow: A list of user ids to follow. :param track: A list of phrases to track. :param locations: A list of coordinates to get.

..todo:: Parameter description!

The default access level allows up to 400 track keywords, 5,000 follow userids and 25 0.1-360 degree location boxes. https://dev.twitter.com/docs/streaming-api/methods

poultry.stream.from_twitter_api(target, endpoint, config)[source]

Consume tweets from a Streaming API endpoint.