Dataframes and Reshaping Data

Ky
Last updated last month

Sentenai provides a variety of ways to access your event data for analysis and developing models.

Get data for a range

Sentenai's range method is a way to get all events within a certain time bound. This is a fast way to bulk export data from a stream without processing.

start = datetime(2015,1,1,14)
end = datetime(2015, 1, 2)
hweather.range(start, end)

Dataframe from range, tail and head

Calling the .df() method with return a dataframe for the range of data.

hweather.range(start, end).df()

Re-shaping Dataframe

The .df() method supports arguments which can be used to re-shape the returned dataframe. V is used to indicate a dataframe column that should be modified.

Limiting Columns

The columns to be returned can be passed into the df() method.

hweather.range(start, end).df(V.temperature, V.humidity)

ts

humidity

temperature

2018-10-25 17:27:05

0.44

47.69

2018-10-25 17:28:05

0.44

47.71

2018-10-25 17:29:05

0.44

47.74

Renaming Columns

Columns can be renamed by assigning the column name to a new value. Only columns included in the .df() method will be returned.

hweather.range(start, end).df(Clouds=V.cloudCover, Temp=V.temperature)

ts

Clouds

Temp

2018-10-25 17:27:05

0.38

47.69

2018-10-25 17:28:05

0.38

47.71

2018-10-25 17:29:05

0.38

47.72

Modifying Column Values

Formulas can be applied to each item in a column. For example, we can show temperature in both Fahrenheit and Celsius.

hweather.range(start, end).df(TF=V.temperature, TC=((V.temperature - 32) * (5/9)))

ts

TF

TC

2018-10-25 17:27:05

47.69

8.71

2018-10-25 17:28:05

47.71

8.72

2018-10-25 17:29:05

47.72

8.73

Inserting a Column

Columns with a constant value can be inserted with the .df() method.

hweather.range(start, end).df(Temp=V.temperature, Unit="F")

ts

Temp

Unit

2018-10-25 17:27:05

47.69

F

2018-10-25 17:28:05

47.71

F

2018-10-25 17:29:05

47.72

F

‚Äč