NCBI C++ ToolKit
Public Types | Public Member Functions | List of all members
bm::aggregator< BV >::pipeline< Opt > Class Template Reference

Search Toolkit Book for bm::aggregator::pipeline

Pipeline vector for running a group of aggregation operations on a family of vectors. More...

#include <util/bitset/bmaggregator.h>

+ Collaboration diagram for bm::aggregator< BV >::pipeline< Opt >:

Public Types

typedef Opt options_type
 

Public Member Functions

 pipeline ()
 
run_optionsoptions () noexcept
 Set pipeline run options. More...
 
const run_optionsget_options () const noexcept
 Get pipeline run options. More...
 
Aggregator traits and control constants
 ~pipeline () noexcept
 
pipeline argument groups fill-in methods
arg_groupsadd ()
 Add new arguments group. More...
 
void set_or_target (bvector_type *bv_or) noexcept
 Attach OR (aggregator vector). More...
 
void set_search_count_limit (size_type limit) noexcept
 Set search limit for results. More...
 
void complete ()
 Prepare pipeline for the execution (resize and init internal structures) Once complete, you cannot add() to it. More...
 
bool is_complete () const noexcept
 return true if pipeline is ready for execution (complete) More...
 
size_type size () const noexcept
 Return size() of pileine. More...
 
const arg_vector_typeget_args_vector () const noexcept
 Return argument vector used for pipeline execution. More...
 
bvect_vector_typeget_bv_res_vector () noexcept
 Return results vector used for pipeline execution. More...
 
bv_count_vector_typeget_bv_count_vector () noexcept
 Return results vector count used for pipeline execution. More...
 

access to internals

class aggregator
 
run_options options_
 execution parameters More...
 
bool is_complete_ = false
 ready to run state flag More...
 
size_t total_vect_ = 0
 total number of vector mentions in all groups More...
 
arg_vector_type arg_vect_
 input arg. groups More...
 
bvect_vector_type bv_res_vect_
 results (bit-vector ptrs) More...
 
bv_count_vector_type count_res_vect_
 results (counts) More...
 
size_type search_count_limit_ {bm::id_max}
 search limit by count More...
 
pipeline_bcache bcache_
 blocks cache structure More...
 
unsigned top_blocks_ = 1
 top-level structure size, max of all bvectors More...
 
bvector_typebv_or_target_ = 0
 OR target bit-bector ptr. More...
 
const bv_vector_typeget_all_input_vect () const noexcept
 
const count_vector_typeget_all_input_cnt_vect () const noexcept
 
size_t unique_vectors () const noexcept
 Return number of unique vectors in the pipeline (after complete()) More...
 
size_t compute_run_batch () const noexcept
 Function looks at the pipeline to apply euristics to suggest optimal run_batch parameter. More...
 
pipeline_bcacheget_bcache () noexcept
 
unsigned get_top_blocks () const noexcept
 Return number of top blocks after complete. More...
 
 pipeline (const pipeline &)=delete
 
pipelineoperator= (const pipeline &)=delete
 
void complete_arg_group (arg_groups *ag)
 
void complete_arg_sub_group (index_vector_type &idx_vect, const bvector_type_const_ptr *bv_src, size_t size)
 

Detailed Description

template<typename BV>
template<class Opt = bm::agg_run_options<>>
class bm::aggregator< BV >::pipeline< Opt >

Pipeline vector for running a group of aggregation operations on a family of vectors.

Pipeline is used to run multiple aggregation combinations (searches) for essencially same set of vectors (different combinations of ANDs and SUBs for example). Pipeline execution improves CPU cache reuse and caches the compressed blocks to re-use it for more efficient execution. Essencially it is a tool to run thousads of runs at once faster.

Definition at line 223 of file bmaggregator.h.

Member Typedef Documentation

◆ options_type

template<typename BV >
template<class Opt = bm::agg_run_options<>>
typedef Opt bm::aggregator< BV >::pipeline< Opt >::options_type

Definition at line 226 of file bmaggregator.h.

Constructor & Destructor Documentation

◆ pipeline() [1/2]

template<typename BV >
template<class Opt = bm::agg_run_options<>>
bm::aggregator< BV >::pipeline< Opt >::pipeline ( )
inline

Definition at line 228 of file bmaggregator.h.

◆ ~pipeline()

template<typename BV >
template<class Opt >
bm::aggregator< BV >::pipeline< Opt >::~pipeline
noexcept

Definition at line 2761 of file bmaggregator.h.

References bm::aligned_free(), arr, and i.

◆ pipeline() [2/2]

template<typename BV >
template<class Opt = bm::agg_run_options<>>
bm::aggregator< BV >::pipeline< Opt >::pipeline ( const pipeline< Opt > &  )
protecteddelete

Member Function Documentation

◆ add()

template<typename BV >
template<class Opt >
aggregator< BV >::arg_groups * bm::aggregator< BV >::pipeline< Opt >::add

Add new arguments group.

Definition at line 2888 of file bmaggregator.h.

References BM_ASSERT.

Referenced by AggregatorTest(), StressTestAggregatorAND(), and StressTestAggregatorAND_SUB().

◆ complete()

template<typename BV >
template<class Opt >
void bm::aggregator< BV >::pipeline< Opt >::complete

◆ complete_arg_group()

template<typename BV >
template<class Opt >
void bm::aggregator< BV >::pipeline< Opt >::complete_arg_group ( arg_groups ag)
private

◆ complete_arg_sub_group()

template<typename BV >
template<class Opt >
void bm::aggregator< BV >::pipeline< Opt >::complete_arg_sub_group ( index_vector_type idx_vect,
const bvector_type_const_ptr bv_src,
size_t  size 
)
private

◆ compute_run_batch()

template<typename BV >
template<class Opt >
size_t bm::aggregator< BV >::pipeline< Opt >::compute_run_batch
noexcept

Function looks at the pipeline to apply euristics to suggest optimal run_batch parameter.

Definition at line 2899 of file bmaggregator.h.

References bm::set_block_size.

◆ get_all_input_cnt_vect()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
const count_vector_type& bm::aggregator< BV >::pipeline< Opt >::get_all_input_cnt_vect ( ) const
inlinenoexcept

Definition at line 297 of file bmaggregator.h.

Referenced by AggregatorTest().

◆ get_all_input_vect()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
const bv_vector_type& bm::aggregator< BV >::pipeline< Opt >::get_all_input_vect ( ) const
inlinenoexcept

Definition at line 295 of file bmaggregator.h.

Referenced by AggregatorTest().

◆ get_args_vector()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
const arg_vector_type& bm::aggregator< BV >::pipeline< Opt >::get_args_vector ( ) const
inlinenoexcept

Return argument vector used for pipeline execution.

Definition at line 279 of file bmaggregator.h.

Referenced by AggregatorTest().

◆ get_bcache()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
pipeline_bcache& bm::aggregator< BV >::pipeline< Opt >::get_bcache ( )
inlineprotectednoexcept

Definition at line 310 of file bmaggregator.h.

◆ get_bv_count_vector()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
bv_count_vector_type& bm::aggregator< BV >::pipeline< Opt >::get_bv_count_vector ( )
inlinenoexcept

Return results vector count used for pipeline execution.

Definition at line 287 of file bmaggregator.h.

Referenced by AggregatorTest().

◆ get_bv_res_vector()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
bvect_vector_type& bm::aggregator< BV >::pipeline< Opt >::get_bv_res_vector ( )
inlinenoexcept

Return results vector used for pipeline execution.

Definition at line 283 of file bmaggregator.h.

Referenced by AggregatorTest(), StressTestAggregatorAND(), and StressTestAggregatorAND_SUB().

◆ get_options()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
const run_options& bm::aggregator< BV >::pipeline< Opt >::get_options ( ) const
inlinenoexcept

Get pipeline run options.

Definition at line 235 of file bmaggregator.h.

◆ get_top_blocks()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
unsigned bm::aggregator< BV >::pipeline< Opt >::get_top_blocks ( ) const
inlineprotectednoexcept

Return number of top blocks after complete.

Definition at line 315 of file bmaggregator.h.

◆ is_complete()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
bool bm::aggregator< BV >::pipeline< Opt >::is_complete ( ) const
inlinenoexcept

return true if pipeline is ready for execution (complete)

Definition at line 269 of file bmaggregator.h.

Referenced by AggregatorTest().

◆ operator=()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
pipeline& bm::aggregator< BV >::pipeline< Opt >::operator= ( const pipeline< Opt > &  )
protecteddelete

◆ options()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
run_options& bm::aggregator< BV >::pipeline< Opt >::options ( )
inlinenoexcept

Set pipeline run options.

Definition at line 232 of file bmaggregator.h.

◆ set_or_target()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
void bm::aggregator< BV >::pipeline< Opt >::set_or_target ( bvector_type bv_or)
inlinenoexcept

Attach OR (aggregator vector).

Pipeline results all will be OR-ed (UNION) into the OR target vector

Parameters
bv_or- OR target vector

Definition at line 251 of file bmaggregator.h.

Referenced by StressTestAggregatorAND(), and StressTestAggregatorAND_SUB().

◆ set_search_count_limit()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
void bm::aggregator< BV >::pipeline< Opt >::set_search_count_limit ( size_type  limit)
inlinenoexcept

Set search limit for results.

Requires that bit-counting to be enabled in the template parameters. Warning: search limit is approximate (for performance reasons) so it can occasinally find more than requested. It cannot find less.

Parameters
limit- search limit (target population count to search for)

Definition at line 260 of file bmaggregator.h.

◆ size()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
size_type bm::aggregator< BV >::pipeline< Opt >::size ( void  ) const
inlinenoexcept

Return size() of pileine.

Definition at line 272 of file bmaggregator.h.

◆ unique_vectors()

template<typename BV >
template<class Opt = bm::agg_run_options<>>
size_t bm::aggregator< BV >::pipeline< Opt >::unique_vectors ( ) const
inlinenoexcept

Return number of unique vectors in the pipeline (after complete())

Definition at line 301 of file bmaggregator.h.

Friends And Related Function Documentation

◆ aggregator

template<typename BV >
template<class Opt = bm::agg_run_options<>>
friend class aggregator
friend

Definition at line 323 of file bmaggregator.h.

Member Data Documentation

◆ arg_vect_

template<typename BV >
template<class Opt = bm::agg_run_options<>>
arg_vector_type bm::aggregator< BV >::pipeline< Opt >::arg_vect_
protected

◆ bcache_

template<typename BV >
template<class Opt = bm::agg_run_options<>>
pipeline_bcache bm::aggregator< BV >::pipeline< Opt >::bcache_
protected

◆ bv_or_target_

template<typename BV >
template<class Opt = bm::agg_run_options<>>
bvector_type* bm::aggregator< BV >::pipeline< Opt >::bv_or_target_ = 0
protected

OR target bit-bector ptr.

Definition at line 340 of file bmaggregator.h.

Referenced by bm::aggregator< bvector_type >::pipeline::set_or_target().

◆ bv_res_vect_

template<typename BV >
template<class Opt = bm::agg_run_options<>>
bvect_vector_type bm::aggregator< BV >::pipeline< Opt >::bv_res_vect_
protected

results (bit-vector ptrs)

Definition at line 334 of file bmaggregator.h.

Referenced by bm::aggregator< bvector_type >::pipeline::get_bv_res_vector().

◆ count_res_vect_

template<typename BV >
template<class Opt = bm::agg_run_options<>>
bv_count_vector_type bm::aggregator< BV >::pipeline< Opt >::count_res_vect_
protected

results (counts)

Definition at line 335 of file bmaggregator.h.

Referenced by bm::aggregator< bvector_type >::pipeline::get_bv_count_vector().

◆ is_complete_

template<typename BV >
template<class Opt = bm::agg_run_options<>>
bool bm::aggregator< BV >::pipeline< Opt >::is_complete_ = false
protected

ready to run state flag

Definition at line 330 of file bmaggregator.h.

Referenced by bm::aggregator< bvector_type >::pipeline::is_complete().

◆ options_

template<typename BV >
template<class Opt = bm::agg_run_options<>>
run_options bm::aggregator< BV >::pipeline< Opt >::options_
protected

◆ search_count_limit_

template<typename BV >
template<class Opt = bm::agg_run_options<>>
size_type bm::aggregator< BV >::pipeline< Opt >::search_count_limit_ {bm::id_max}
protected

search limit by count

Definition at line 336 of file bmaggregator.h.

Referenced by bm::aggregator< bvector_type >::pipeline::set_search_count_limit().

◆ top_blocks_

template<typename BV >
template<class Opt = bm::agg_run_options<>>
unsigned bm::aggregator< BV >::pipeline< Opt >::top_blocks_ = 1
protected

top-level structure size, max of all bvectors

Definition at line 339 of file bmaggregator.h.

Referenced by bm::aggregator< bvector_type >::pipeline::get_top_blocks().

◆ total_vect_

template<typename BV >
template<class Opt = bm::agg_run_options<>>
size_t bm::aggregator< BV >::pipeline< Opt >::total_vect_ = 0
protected

total number of vector mentions in all groups

Definition at line 331 of file bmaggregator.h.


The documentation for this class was generated from the following file:
Modified on Fri Sep 20 14:57:25 2024 by modify_doxy.py rev. 669887