Skip to content
Snippets Groups Projects
Commit d277b075 authored by Iustin Pop's avatar Iustin Pop
Browse files

Optimise recursive Query filters


Currently, the And and Or filters use very nice code, e.g. in case of
OrFilter:

   any id <$> mapM evaluateFilter flts

However, looking at the memory profiles shows that application of
any/id to monadic values via '<$>' does not work nicely, losing the
'early' success property. This results in too much memory being used
for thunks in monadic sequencing.

Rather than trying to add more strictness (not sure exactly how, TBH),
switching these to explicit recursion solves the problem, since we
take the 'early' exit problem in our hands and we are explicit about
it.

Memory usage in case of big (e.g. 1000 'Or' elements) is reduced
significantly, and thus also the runtime.

Signed-off-by: default avatarIustin Pop <iustin@google.com>
Reviewed-by: default avatarMichele Tartara <mtartara@google.com>
parent 228ef0f2
No related branches found
No related tags found
No related merge requests found
...@@ -144,14 +144,28 @@ containsFilter (NumericValue val) lst = do ...@@ -144,14 +144,28 @@ containsFilter (NumericValue val) lst = do
-- | Verifies if a given item passes a filter. The runtime context -- | Verifies if a given item passes a filter. The runtime context
-- might be missing, in which case most of the filters will consider -- might be missing, in which case most of the filters will consider
-- this as passing the filter. -- this as passing the filter.
--
-- Note: we use explicit recursion to reduce unneeded memory use;
-- 'any' and 'all' do not play nice with monadic values, resulting in
-- either too much memory use or in too many thunks being created.
evaluateFilter :: ConfigData -> Maybe b -> a evaluateFilter :: ConfigData -> Maybe b -> a
-> Filter (FieldGetter a b) -> Filter (FieldGetter a b)
-> ErrorResult Bool -> ErrorResult Bool
evaluateFilter _ _ _ EmptyFilter = Ok True evaluateFilter _ _ _ EmptyFilter = Ok True
evaluateFilter c mb a (AndFilter flts) = evaluateFilter c mb a (AndFilter flts) = helper flts
all id <$> mapM (evaluateFilter c mb a) flts where helper [] = Ok True
evaluateFilter c mb a (OrFilter flts) = helper (f:fs) = do
any id <$> mapM (evaluateFilter c mb a) flts v <- evaluateFilter c mb a f
if v
then helper fs
else Ok False
evaluateFilter c mb a (OrFilter flts) = helper flts
where helper [] = Ok False
helper (f:fs) = do
v <- evaluateFilter c mb a f
if v
then Ok True
else helper fs
evaluateFilter c mb a (NotFilter flt) = evaluateFilter c mb a (NotFilter flt) =
not <$> evaluateFilter c mb a flt not <$> evaluateFilter c mb a flt
evaluateFilter c mb a (TrueFilter getter) = evaluateFilter c mb a (TrueFilter getter) =
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment