datadiligence.evaluators package

Submodules

datadiligence.evaluators.base module

This module contains the base Evaluator class.

class datadiligence.evaluators.base.Evaluator[source]

Bases: object

Base class for evaluators. is_allowed must be implemented.

add_rule(rule)[source]

Add a rule to the evaluator.

filter_allowed(**kwargs)[source]

Filter a list of entries based on the rules in this evaluator.

is_allowed(**kwargs)[source]

Check each rule to see if the request is allowed.

Parameters:

**kwargs (any) – Keyword args to pass to rule

Returns:

True if the content is allowed, False otherwise.

Return type:

bool

name = 'base_evaluator'

datadiligence.evaluators.file_evaluator module

FileEvaluator evaluator module.

class datadiligence.evaluators.file_evaluator.FileEvaluator[source]

Bases: Evaluator

File Evaluator class. Loads C2PAMetadataRule rule by default.

is_allowed(**kwargs)[source]

Check if the headers are allowed based on the rules in this evaluator.

Parameters:

**path (str) – The path of the file.

Returns:

True if the content is allowed, False otherwise.

Return type:

bool

name = 'file'

datadiligence.evaluators.http module

This module contains the HttpEvaluator class.

class datadiligence.evaluators.http.HttpEvaluator(user_agent=None, respect_robots=True, respect_tdmrep=True)[source]

Bases: Evaluator

HTTP Evaluator class. Loads XRobotsTagHeader rule by default.

name = 'http'

datadiligence.evaluators.postprocess module

Postprocess evaluator module.

class datadiligence.evaluators.postprocess.PostprocessEvaluator(user_agent=None)[source]

Bases: Evaluator

Postprocess Evaluator class. Loads XRobotsTagHeader, TDMRepHeader, and C2PAMetadata rules by default.

is_allowed(**kwargs)[source]

Check if the headers are allowed based on the rules in this evaluator.

Parameters:
Returns:

True if the content is allowed, False otherwise.

Return type:

bool

name = 'postprocess'

datadiligence.evaluators.preprocess module

This module contains the PreprocessEvaluator class.

class datadiligence.evaluators.preprocess.PreprocessEvaluator(user_agent=None)[source]

Bases: Evaluator

Preprocess Evaluator class. Loads SpawningAPI rule by default.

add_rule(rule)[source]

Add a rule to the evaluator.

filter_allowed(urls=None, **kwargs)[source]

Filter a list of urls based on the rules in this evaluator.

Parameters:

  • urls (list) – A list of urls to filter.

  • **kwargs – Arbitrary keyword arguments to read args from.

Returns:

A list of urls that are allowed.

Return type:

list

is_allowed(urls=None, **kwargs)[source]

Check if the urls are allowed.

Parameters:

  • urls (list) – A list of urls to check.

  • **kwargs – Arbitrary keyword arguments to read args from.

Returns:

List of boolean values, respectively indicating if can be used or not

Return type:

bool

name = 'preprocess'

Module contents

This module contains default Evaluators.