souffle::ReadStreamCSV Class Reference

#include <ReadStreamCSV.h>

Inheritance diagram for souffle::ReadStreamCSV:

Collaboration diagram for souffle::ReadStreamCSV:

Public Member Functions
	ReadStreamCSV (std::istream &file, const std::map< std::string, std::string > &rwOperation, SymbolTable &symbolTable, RecordTable &recordTable)
Public Member Functions inherited from souffle::ReadStream
template<typename T >
void	readAll (T &relation)
Public Member Functions inherited from souffle::SerialisationStream< false >
virtual	~SerialisationStream ()=default

Protected Member Functions
std::map< int, int >	getInputColumnMap (const std::map< std::string, std::string > &rwOperation, const unsigned arity_) const
std::string	nextElement (const std::string &line, size_t &start, size_t &end)
Own< RamDomain[]>	readNextTuple () override
	Read and return the next tuple. More...
RamUnsigned	readRamUnsigned (const std::string &element, size_t &charactersRead)
	Read an unsigned element. More...
Protected Member Functions inherited from souffle::ReadStream
void	consumeChar (const std::string &str, char c, size_t &pos)
	Read past given character, consuming any preceding whitespace. More...
void	consumeWhiteSpace (const std::string &str, size_t &pos)
	Advance position in the string until first non-whitespace character. More...
RamDomain	readADT (const std::string &source, const std::string &adtName, size_t pos=0, size_t *charactersRead=nullptr)
std::string	readAlphanumeric (const std::string &source, size_t &pos)
	Read the next alphanumeric sequence (corresponding to IDENT). More...
RamDomain	readRecord (const std::string &source, const std::string &recordTypeName, size_t pos=0, size_t *charactersRead=nullptr)
	Read a record from a string. More...
	ReadStream (const std::map< std::string, std::string > &rwOperation, SymbolTable &symTab, RecordTable &recTab)
std::string	readUntil (const std::string &source, const std::string stopChars, const size_t pos, size_t *charactersRead)
Protected Member Functions inherited from souffle::SerialisationStream< false >
	SerialisationStream (RO< SymbolTable > &symTab, RO< RecordTable > &recTab, const std::map< std::string, std::string > &rwOperation)
	SerialisationStream (RO< SymbolTable > &symTab, RO< RecordTable > &recTab, Json types)
	SerialisationStream (RO< SymbolTable > &symTab, RO< RecordTable > &recTab, Json types, std::vector< std::string > relTypes, size_t auxArity=0)

Protected Attributes
const std::string	delimiter
std::istream &	file
std::map< int, int >	inputMap
size_t	lineNumber
Protected Attributes inherited from souffle::SerialisationStream< false >
size_t	arity
size_t	auxiliaryArity
RO< RecordTable > &	recordTable
RO< SymbolTable > &	symbolTable
std::vector< std::string >	typeAttributes
Json	types

Additional Inherited Members
Protected Types inherited from souffle::SerialisationStream< false >
using	RO = std::conditional_t< readOnlyTables, const A, A >

Detailed Description

Definition at line 49 of file ReadStreamCSV.h.

Constructor & Destructor Documentation

ReadStreamCSV()

souffle::ReadStreamCSV::ReadStreamCSV ( std::istream &  file, const std::map< std::string, std::string > &  rwOperation, SymbolTable &  symbolTable, RecordTable &  recordTable  )

inline

Definition at line 51 of file ReadStreamCSV.h.

                                                                                         {
        while (inputMap.size() < arity) {
            int size = static_cast<int>(inputMap.size());
            inputMap[size] = size;
        }
    }
 
protected:
    /**
     * Read and return the next tuple.

References souffle::SerialisationStream< false >::arity, inputMap, and TCB_SPAN_NAMESPACE_NAME::detail::size().

Here is the call graph for this function:

Member Function Documentation

getInputColumnMap()

std::map<int, int> souffle::ReadStreamCSV::getInputColumnMap ( const std::map< std::string, std::string > &  rwOperation, const unsigned  arity_  ) const

inlineprotected

Definition at line 217 of file ReadStreamCSV.h.

                                   {
            std::istringstream iss(columnString);
            std::string mapping;
            int index = 0;
            while (std::getline(iss, mapping, ':')) {
                inputColumnMap[stoi(mapping)] = index++;
            }
            if (inputColumnMap.size() < arity_) {
                throw std::invalid_argument("Invalid column set was given: <" + columnString + ">");
            }
        } else {
            while (inputColumnMap.size() < arity_) {
                int size = static_cast<int>(inputColumnMap.size());
                inputColumnMap[size] = size;
            }
        }
        return inputColumnMap;
    }
 
    const std::string delimiter;
    std::istream& file;
    size_t lineNumber;

nextElement()

std::string souffle::ReadStreamCSV::nextElement ( const std::string &  line, size_t &  start, size_t &  end  )

inlineprotected

Definition at line 163 of file ReadStreamCSV.h.

                                                  {
            int record_parens = 0;
            size_t next_delimiter = line.find(delimiter, start);
 
            // Find first delimiter after the record.
            while (end < std::min(next_delimiter, line.length()) || record_parens != 0) {
                // Track the number of parenthesis.
                if (line[end] == '[') {
                    ++record_parens;
                } else if (line[end] == ']') {
                    --record_parens;
                }
 
                // Check for unbalanced parenthesis.
                if (record_parens < 0) {
                    break;
                };
 
                ++end;
 
                // Find a next delimiter if the old one is invalid.
                // But only if inside the unbalance parenthesis.
                if (end == next_delimiter && record_parens != 0) {
                    next_delimiter = line.find(delimiter, end);
                }
            }
 
            // Handle the end-of-the-line case where parenthesis are unbalanced.
            if (record_parens != 0) {
                std::stringstream errorMessage;
                errorMessage << "Unbalanced record parenthesis " << lineNumber << "; ";
                throw std::invalid_argument(errorMessage.str());
            }
        } else {
            end = std::min(line.find(delimiter, start), line.length());
        }
 
        // Check for missing value.
        if (start > end) {
            std::stringstream errorMessage;
            errorMessage << "Values missing in line " << lineNumber << "; ";
            throw std::invalid_argument(errorMessage.str());
        }
 
        element = line.substr(start, end - start);
        start = end + delimiter.size();
 
        return element;
    }
 
    std::map<int, int> getInputColumnMap(
            const std::map<std::string, std::string>& rwOperation, const unsigned arity_) const {
        std::string columnString = getOr(rwOperation, "columns", "");

References delimiter, and lineNumber.

readNextTuple()

Own<RamDomain[]> souffle::ReadStreamCSV::readNextTuple ( )

inlineoverrideprotectedvirtual

Read and return the next tuple.

Returns nullptr if no tuple was readable.

Returns

Implements souffle::ReadStream.

Reimplemented in souffle::ReadFileCSV.

Definition at line 69 of file ReadStreamCSV.h.

                                  {
            return nullptr;
        }
        // Handle Windows line endings on non-Windows systems
        if (!line.empty() && line.back() == '\r') {
            line = line.substr(0, line.length() - 1);
        }
        ++lineNumber;
 
        size_t start = 0;
        size_t end = 0;
        size_t columnsFilled = 0;
        for (uint32_t column = 0; columnsFilled < arity; column++) {
            size_t charactersRead = 0;
            std::string element = nextElement(line, start, end);
            if (inputMap.count(column) == 0) {
                continue;
            }
            ++columnsFilled;
 
            try {
                auto&& ty = typeAttributes.at(inputMap[column]);
                switch (ty[0]) {
                    case 's': {
                        tuple[inputMap[column]] = symbolTable.unsafeLookup(element);
                        charactersRead = element.size();
                        break;
                    }
                    case 'r': {
                        tuple[inputMap[column]] = readRecord(element, ty, 0, &charactersRead);
                        break;
                    }
                    case '+': {
                        tuple[inputMap[column]] = readADT(element, ty, 0, &charactersRead);
                        break;
                    }
                    case 'i': {
                        tuple[inputMap[column]] = RamSignedFromString(element, &charactersRead);
                        break;
                    }
                    case 'u': {
                        tuple[inputMap[column]] = ramBitCast(readRamUnsigned(element, charactersRead));
                        break;
                    }
                    case 'f': {
                        tuple[inputMap[column]] = ramBitCast(RamFloatFromString(element, &charactersRead));
                        break;
                    }
                    default: fatal("invalid type attribute: `%c`", ty[0]);
                }
                // Check if everything was read.
                if (charactersRead != element.size()) {
                    throw std::invalid_argument(
                            "Expected: " + delimiter + " or \\n. Got: " + element[charactersRead]);
                }
            } catch (...) {
                std::stringstream errorMessage;
                errorMessage << "Error converting <" + element + "> in column " << column + 1 << " in line "
                             << lineNumber << "; ";
                throw std::invalid_argument(errorMessage.str());
            }
        }
 
        return tuple;
    }
 
    /**
     * Read an unsigned element. Possible bases are 2, 10, 16
     * Base is indicated by the first two chars.

readRamUnsigned()

RamUnsigned souffle::ReadStreamCSV::readRamUnsigned ( const std::string &  element, size_t &  charactersRead  )

inlineprotected

Read an unsigned element.

Possible bases are 2, 10, 16 Base is indicated by the first two chars.

Definition at line 146 of file ReadStreamCSV.h.

                                     {
            value = RamUnsignedFromString(element, &charactersRead, 2);
        } else if (isPrefix("0x", element)) {
            value = RamUnsignedFromString(element, &charactersRead, 16);
        } else {
            value = RamUnsignedFromString(element, &charactersRead);
        }
        return value;
    }
 
    std::string nextElement(const std::string& line, size_t& start, size_t& end) {
        std::string element;
 

References souffle::RamUnsignedFromString().

Here is the call graph for this function:

Field Documentation

delimiter

const std::string souffle::ReadStreamCSV::delimiter

protected

Definition at line 241 of file ReadStreamCSV.h.

Referenced by nextElement().

file

std::istream& souffle::ReadStreamCSV::file

protected

Definition at line 242 of file ReadStreamCSV.h.

Referenced by souffle::ReadFileCSV::ReadFileCSV().

inputMap

std::map<int, int> souffle::ReadStreamCSV::inputMap

protected

Definition at line 244 of file ReadStreamCSV.h.

Referenced by ReadStreamCSV().

lineNumber

size_t souffle::ReadStreamCSV::lineNumber

protected

Definition at line 243 of file ReadStreamCSV.h.

Referenced by nextElement().

---

The documentation for this class was generated from the following file:

• include/souffle/io/ReadStreamCSV.h