Class: CharDet::CharDistributionAnalysis

### Home Classes Methods

### In Files

- lib/tmail/vendor/rchardet-1.3/lib/rchardet/chardistribution.rb

### Parent

Object

### Methods

- ::new
- #feed
- #get_confidence
- #get_order
- #got_enough_data
- #reset

### Files

- CHANGES
- LICENSE
- NOTES
- README
- Rakefile
- MANIFEST
- Makefile
- depend
- Makefile
- COPYING
- README

### Class/Module Index

Quicksearch

- TMail
- TMail::Address
- TMail::AddressGroup
- TMail::AddressHeader
- TMail::Attachment
- TMail::ContentDispositionHeader
- TMail::ContentTransferEncodingHeader
- TMail::ContentTypeHeader
- TMail::DateTimeHeader
- TMail::EncryptedHeader
- TMail::FilePort
- TMail::HeaderField
- TMail::KeywordsHeader
- TMail::Mail
- TMail::MailFlags
- TMail::Maildir
- TMail::MaildirLoader
- TMail::MaildirPort
- TMail::MboxLoader
- TMail::MessageIdHeader
- TMail::MhLoader
- TMail::MhMailbox
- TMail::MhPort
- TMail::MimeVersionHeader
- TMail::Parser
- TMail::Port
- TMail::ReceivedHeader
- TMail::ReferencesHeader
- TMail::ReturnPathHeader
- TMail::SingleAddressHeader
- TMail::StringPort
- TMail::StructuredHeader
- TMail::SyntaxError
- TMail::TMailScanner
- TMail::TextUtils
- TMail::UNIXMbox
- TMail::UnstructuredHeader
- CharDet
- CharDet::Big5DistributionAnalysis
- CharDet::Big5Prober
- CharDet::CharDistributionAnalysis
- CharDet::CharSetGroupProber
- CharDet::CharSetProber
- CharDet::CodingStateMachine
- CharDet::EUCJPContextAnalysis
- CharDet::EUCJPDistributionAnalysis
- CharDet::EUCJPProber
- CharDet::EUCKRDistributionAnalysis
- CharDet::EUCKRProber
- CharDet::EUCTWDistributionAnalysis
- CharDet::EUCTWProber
- CharDet::EscCharSetProber
- CharDet::GB2312DistributionAnalysis
- CharDet::GB2312Prober
- CharDet::HebrewProber
- CharDet::JapaneseContextAnalysis
- CharDet::Latin1Prober
- CharDet::MBCSGroupProber
- CharDet::MultiByteCharSetProber
- CharDet::SBCSGroupProber
- CharDet::SJISContextAnalysis
- CharDet::SJISDistributionAnalysis
- CharDet::SJISProber
- CharDet::SingleByteCharSetProber
- CharDet::UTF8Prober
- CharDet::UniversalDetector

No matching classes.

# CharDet::CharDistributionAnalysis

### Public Class Methods

new() click to toggle source

    # File lib/tmail/vendor/rchardet-1.3/lib/rchardet/chardistribution.rb, line 36
    def initialize
      @_mCharToFreqOrder = nil # Mapping table to get frequency order from char order (get from GetOrder())
      @_mTableSize = nil # Size of above table
      @_mTypicalDistributionRatio = nil # This is a constant value which varies from language to language, used in calculating confidence.  See http://www.mozilla.org/projects/intl/UniversalCharsetDetection.html for further detail.
      reset()
    end

### Public Instance Methods

feed(aStr, aCharLen) click to toggle source

    # File lib/tmail/vendor/rchardet-1.3/lib/rchardet/chardistribution.rb, line 50
    def feed(aStr, aCharLen)
      # # """feed a character with known length"""
      if aCharLen == 2
        # we only care about 2-bytes character in our distribution analysis
        order = get_order(aStr)
      else
        order = -1
      end
      if order >= 0
        @_mTotalChars += 1
        # order is valid
        if order < @_mTableSize
          if 512 > @_mCharToFreqOrder[order]
            @_mFreqChars += 1
          end
        end
      end
    end

get_confidence() click to toggle source

    # File lib/tmail/vendor/rchardet-1.3/lib/rchardet/chardistribution.rb, line 69
    def get_confidence
      # """return confidence based on existing data"""
      # if we didn't receive any character in our consideration range, return negative answer
      if @_mTotalChars <= 0
        return SURE_NO
      end

      if @_mTotalChars != @_mFreqChars
        r = @_mFreqChars / ((@_mTotalChars - @_mFreqChars) * @_mTypicalDistributionRatio)
        if r < SURE_YES
          return r
        end
      end

      # normalize confidence (we don't want to be 100% sure)
      return SURE_YES
    end

get_order(aStr) click to toggle source

    # File lib/tmail/vendor/rchardet-1.3/lib/rchardet/chardistribution.rb, line 93
    def get_order(aStr)
      # We do not handle characters based on the original encoding string, but
      # convert this encoding string to a number, here called order.
      # This allows multiple encodings of a language to share one frequency table.
      return -1
    end

got_enough_data() click to toggle source

    # File lib/tmail/vendor/rchardet-1.3/lib/rchardet/chardistribution.rb, line 87
    def got_enough_data
      # It is not necessary to receive all data to draw conclusion. For charset detection,
      # certain amount of data is enough
      return @_mTotalChars > ENOUGH_DATA_THRESHOLD
    end

reset() click to toggle source

    # File lib/tmail/vendor/rchardet-1.3/lib/rchardet/chardistribution.rb, line 43
    def reset
      # # """reset analyser, clear any state"""
      @_mDone = false # If this flag is set to constants.True, detection is done and conclusion has been made
      @_mTotalChars = 0 # Total characters encountered
      @_mFreqChars = 0 # The number of characters whose frequency order is less than 512
    end

[Validate]

Generated with the Darkfish Rdoc Generator 2.