NAME¶

ngramshrink - Shrink ngram model from input model file.

DESCRIPTION¶

Shrink ngram model from input model file.

SYNOPSIS¶

ngramshrink [--options] [in.fst [out.fst]]

OPTIONS¶

--backoff_label: type = int64, default = 0

Backoff label

--check_consistency: type = bool, default = false

Check model consistency

--context_pattern: type = string, default = ""

Pattern of contexts to prune

--count_pattern: type = string, default = ""

Pattern of counts to prune

--method: type = string, default = "seymore"

One of: "context_prune", "count_prune", "relative_entropy", "seymore"

--norm_eps: type = double, default = 0.001

Normalization check epsilon

--shrink_opt: type = int32, default = 0

Optimization level: Range 0 (fastest) to 2 (most accurate)

--target_number_of_ngrams: type = int64, default = -1

Maximum number of ngrams to leave in model after pruning. Value less than zero means no target number, just use theta.

--theta: type = double, default = 0

Pruning threshold theta

--total_unigram_count: type = double, default = -1

Total unigram count

--help: type = bool, default = false

show usage information

--helpshort: type = bool, default = false

show brief usage information

--tmpdir: type = string, default = "/tmp"

temporary directory

--v: type = int32, default = 0

verbosity level

--fst_align: type = bool, default = false

Write FST data aligned where appropriate

--fst_default_cache_gc: type = bool, default = true

Enable garbage collection of cache

--fst_default_cache_gc_limit: type = int64, default = 1048576

Cache byte size that triggers garbage collection

--fst_read_mode: type = string, default = "read"

Default file reading mode for mappable files

--fst_verify_properties: type = bool, default = false

Verify FST properties queried by TestProperties

--save_relabel_ipairs: type = string, default = ""

Save input relabel pairs to file

--save_relabel_opairs: type = string, default = ""

Save output relabel pairs to file

--end_symbol: type = string, default = "</s>"

Class label for sentence start

--start_symbol: type = string, default = "<s>"

Class label for sentence start

--fst_compat_symbols: type = bool, default = true

Require symbol tables to match when appropriate

--fst_field_separator: type = string, default = " "

Set of characters used as a separator between printed fields

--fst_error_fatal: type = bool, default = true

FST errors are fatal; o.w. return objects flagged as bad: e.g., FSTs: kError property set, FST weights: not a Member()

--ngram_error_fatal: type = bool, default = true

NGram errors are fatal; o.w. return objects flagged as bad: e.g., NGramModel::Error() returns true.

--fst_weight_parentheses: type = string, default = ""

Characters enclosing the first weight of a printed composite weight (e.g., pair weight, tuple weight and derived classes) to ensure proper I/O of nested composite weights; must have size 0 (none) or 2 (open and close parenthesis)

--fst_weight_separator: type = string, default = ","

Character separator between printed composite weights; must be a single character