Stream Methods

Ky
Last updated last month

Destroying Streams

When you delete an event, it only effects that particular event. If you were to delete a stream, however, it deletes not just the stream but all events contained within. To disambiguate these two actions, deleting a stream users the method destroy, and requires a confirm=True.

test_stream.destory(confirm=True)

Stream Tail and Head

.tail() and .head() will return the head or tail of a specified stream. Both take the optional n for number of items to be returned.

hweather.tail(n=10)
hweather.head(n=10)

Number of Events

len(hweather)

157398

Stream Fields

Calling field on a path will return the unique fields and when they were added.

hweather.fields()

Field

Added At

0

apparentTemperature

2018-10-12 21:19:05+00:00

1

cloudCover

2018-10-12 21:19:05+00:00

2

humidity

2018-10-12 21:19:05+00:00

3

uvIndex

2018-10-12 21:19:05+00:00

Stream Stats and Distribution

.stats returns an object with categorical and numerical keys.

Numerical Stats

On fields containing numerical values, stats will return the max, mean, count, missing, minimum and standard deviation.

hweather.temperature.stats()
{'categorical': {},
'numerical': {'max': 78.68,
'mean': 48.25424832659952,
'count': 16583,
'missing': 0,
'min': -2.49,
'std': 8.193860880038313}}
hweat

Categorical Stats

Fields containing categorical values will return the number of uniques, count, missing, frequency and top value.

hweather.icon.stats()
{'categorical': {'unique': 10,
'count': 16583,
'missing': 0,
'freq': 5010,
'top': 'clear-night'},
'numerical': {}}

Describe

Calling .describe() on a field returns the distribution for a field.

Numerical Fields

hweather.temperature.describe()
"count" 16583
"mean" 48.25
"std" 8.19
"min" -2.49
"25%" None
"50%" None
"75%" None
"max" 78.68

Categorical Fields

hweather.icon.describe()
"count" 16583
"unique" 10
"top" clear-night
"freq" 5010

Getting Values

.values returns a Values object with the most recent values for each field in a stream.

hweather.values()

Event Path

Value

Updated At

0

apparentTemperature

37.47

2017-12-04 04:00:00+00:00

1

cloudCover

0.87

2017-12-04 04:00:00+00:00

2

dewPoint

30.4

2017-12-04 04:00:00+00:00

3

humidity

0.75

2017-12-04 04:00:00+00:00

...

...

...

...

15

precipAccumulation

0.012

2017-11-28 12:00:00+00:00

16

precipType

rain

2017-12-03 19:00:00+00:00

17

windGust

1.64

2017-12-04 12:00:00+00:00

Passing in an optional at argument will return all values at that time for a stream.

hweather.values(at=datetime(2015,1,1)

Event Path

Value

Updated At

0

apparentTemperature

21.27

2015-01-01 00:00:00+00:00

1

cloudCover

0.27

2014-12-31 23:00:00+00:00

2

dewPoint

11.8

2015-01-01 00:00:00+00:00

3

humidity

0.47

2015-01-01 00:00:00+00:00

...

...

...

...

14

precipAccumulation

0.033

2014-12-21 00:00:00+00:00

15

precipType

rain

2014-12-28 00:00:00+00:00

A field can be access with it's stream path.

vs = hweather.values()
vs[hweather.humidity]

0.75

Values can be iterated through using the .items method, like a dictionary

for k, v in vs.items():
print("{:40}{}".format(repr(k), v))
apparentTemperature 37.47
cloudCover 0.87
dewPoint 30.4
humidity 0.75
icon partly-cloudy-night
ozone 327.25
precipAccumulation 0.012
precipIntensity 0
precipProbability 0
precipType rain
pressure 1026.49
summary Mostly Cloudy
temperature 37.47
uvIndex 0
visibility 9.77
windBearing 339
windGust 1.64
windSpeed 0.57