pipe-utils¶
Generic piping utilities.
Other functions can be piped to them, from both sides, without having to use the
pipe
object. The resulting function then also inherits this functionality.
Built-in¶
Even though these are defined in the pipetools.utils
module, they’re
importable directly from pipetools
for convenience.
All of these that take a function as an argument can automatically partially apply the given function with positional and/or keyword arguments, for example:
foreach(some_func, foo, bar=None)
Is the same as:
foreach(partial(some_func, foo, bar=None))
(As of 0.1.9
this uses X-partial)
They also automatically convert the X object to an actual function.
List of built-in utils¶
-
pipetools.utils.
as_args
(function)¶ Applies the sequence in the input as positional arguments to function.
some_lists > as_args(izip)
-
pipetools.utils.
as_kwargs
(function)¶ Applies the dictionary in the input as keyword arguments to function.
-
pipetools.utils.
count
¶ Returns the number of items in iterable.
-
pipetools.utils.
debug_print
(function)¶ Prints function applied on input and returns the input.
foo = (pipe | something | debug_print(X.get_status()) | something_else | foreach(debug_print("attr is: {0.attr}")) | etc)
-
pipetools.utils.
drop_first
(count)¶ Assumes an iterable on the input, returns an iterable with identical items except for the first count.
>>> range(10) > drop_first(5) | tuple (5, 6, 7, 8, 9)
-
pipetools.utils.
flatten
¶ Flattens an arbitrarily deep nested iterable(s).
>>> [[[[[[1]]], 2], range(2) > foreach(X + 3)]] > flatten | list [1, 2, 3, 4]
Does not treat strings and (as of
0.3.1
) mappings (dictionaries) as iterables so these are left alone.>>> ('hello', [{'how': 'are'}, [['you']]]) > flatten | list ['hello', {'how': 'are'}, 'you']
Also turns non-iterables into iterables which is convenient when you are not sure about the input.
>>> 'stuff' > flatten | list ['stuff']
-
pipetools.utils.
foreach
(function)¶ Returns a function that takes an iterable and returns an iterator over the results of calling function on each item of the iterable.
>>> range(5) > foreach(factorial) | list [1, 1, 2, 6, 24]
-
pipetools.utils.
foreach_do
(function)¶ Like
foreach()
but is evaluated immediately and doesn’t return anything.For the occasion that you just want to do some side-effects:
open('addresses.txt') > foreach(geocode) | foreach_do(launch_missile)
– With
foreach()
nothing would happen (except an itetrator being created)
-
pipetools.utils.
group_by
(function)¶ Groups input sequence by function.
Returns an iterator over a sequence of tuples where the first item is a result of function and the second one a list of items matching this result.
Ordering of the resulting iterator is undefined, but ordering of the items in the groups is preserved.
>>> [1, 2, 3, 4, 5, 6] > group_by(X % 2) | list [(0, [2, 4, 6]), (1, [1, 3, 5])]
-
pipetools.utils.
select_first
(condition)¶ Returns first item from input sequence that satisfies condition. Or
None
if none does.>>> ['py', 'pie', 'pi'] > select_first(X.startswith('pi')) 'pie'
As of
0.2.1
you can also directly use regular expressions and write the above as:>>> ['py', 'pie', 'pi'] > select_first('^pi') 'pie'
There is also a shortcut for
select_first(X)
calledfirst_of
:>>> first_of(['', None, 0, 3, 'something']) 3 >>> first_of([]) None
-
pipetools.utils.
sort_by
(function)¶ Sorts an incoming sequence by using the given function as key.
>>> range(10) > sort_by(-X) [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
Supports automatic data-structure creation:
users > sort_by([X.last_name, X.first_name])
There is also a shortcut for
sort_by(X)
calledsort
:>>> [4, 5, 8, -3, 0] > sort [-3, 0, 4, 5, 8]
And (as of
0.2.3
) a shortcut for reversing the sort:>>> 'asdfaSfa' > sort_by(X.lower()).descending ['s', 'S', 'f', 'f', 'd', 'a', 'a', 'a']
-
pipetools.utils.
take_first
(count)¶ Assumes an iterable on the input, returns an iterable with first count items from the input (or possibly less, if there isn’t that many).
>>> range(9000) > where(X % 100 == 0) | take_first(5) | tuple (0, 100, 200, 300, 400)
-
pipetools.utils.
take_until
(condition)¶ >>> [1, 4, 6, 4, 1] > take_until(X > 5) | list [1, 4]
>>> [1, 4, 6, 4, 1] > take_until(X > 5).including | list [1, 4, 6]
-
pipetools.utils.
take_until_including
(condition)¶ >>> [1, 4, 6, 4, 1] > take_until_including(X > 5) | list [1, 4, 6]
-
pipetools.utils.
tee
(function)¶ Sends a copy of the input into function - like a T junction.
-
pipetools.utils.
unless
(exception_class_or_tuple, func, *args, **kwargs)¶ When exception_class_or_tuple occurs while executing func, it will be caught and
None
will be returned.>>> f = where(X > 10) | list | unless(IndexError, X[0]) >>> f([5, 8, 12, 4]) 12 >>> f([1, 2, 3]) None
-
pipetools.utils.
where
(condition)¶ Pipe-able lazy filter.
>>> odd_range = range | where(X % 2) | list >>> odd_range(10) [1, 3, 5, 7, 9]
Make your own¶
You can make your own, but you generally shouldn’t need to, since you can pipe any functions you like.
But if you feel like there’s a generally reusable pipe-util missing, feel free to add it via a pull request on github.
How to do it? Just write the function and stick the
pipe_util()
decorator on it.
And optionally also auto_string_formatter()
(see
Automatic string formatting) or
data_structure_builder()
(see Automatic data-structure creation) if it makes sense.
Automatic data-structure creation¶
Some of the utils, most importantly foreach()
, offer a shortcut for
creating basic python data structures - list
, tuple
and dict
.
It works like this (the | list
at the end is just so we can see the result,
otherwise it would just give us <iterable thing at 0xasdf123>
):
>>> range(5) > foreach({X: X * 2}) | list
[{0: 0}, {1: 2}, {2: 4}, {3: 6}, {4: 8}]
>>> range(5) > foreach([X, X * '★']) | list
[[0, ''], [1, '★'], [2, '★★'], [3, '★★★'], [4, '★★★★']]
It can also be combined with string formatting:
>>> names = [('John', 'Matrix'), ('Jack', 'Slater')]
>>> names > foreach({'name': "{0} {1}", 'initials': '{0[0]}. {1[0]}.'}) | list
[{'initials': 'J. M.', 'name': 'John Matrix'},
{'initials': 'J. S.', 'name': 'Jack Slater'}]
Automatic regex conditions¶
If you use a string instead of a function as a condition in
where()
, where_not()
,
select_first()
or take_until()
the
string will be used as a regex to match the input against. This will, of course,
work only if the items of the input sequence are strings.
Essentially:
where(r'^some\-regexp?$')
is equivalent to:
where(re.match, r'^some\-regexp?$')
As of 0.3.2
this also filters out None
values instead of throwing an
exception. Making it equivalent to:
where(maybe | (re.match, r'^some\-regexp?$'))
If you want to easily add this functionality to your own functions, you can use
the regex_condition()
decorator.