![]() You can configure the characters to use as record separators. When you configure the stage, you select the field to use. Text The destination writes data from a single text field to the destination SDC Record The destination writes records in the SDC Record data format. For information about generating the descriptor file, see Protobuf Data Format Prerequisites. Message based on the stage error handling property and continues When the data for a record exceeds 1 MB, the origin cannotĬontinue processing data in the message. Protobuf messages must match the specified message type and beĭescribed in the descriptor file. Origin assumes messages contain multiple protobuf messages. Protobuf Generates a record for every protobuf message. Origin, the origin processes the object based on the error When an object exceeds the maximum object length defined for the You can process JSONįiles that include multiple JSON objects or a single JSONĪrray. JSON Generates a record for each JSON object. Single user-defined escape and quote characters. Multiple user-defined characters to delimit fields and lines, and Multi Character Delimited - File that uses.Custom - File that uses user-defined delimiter,.PostgreSQL Text - PostgreSQL text file.PostgreSQL CSV - PostgreSQL comma-separated.Tab-Separated Values - File that includes.MySQL CSV - MySQL comma-separated file.MS Excel CSV - Microsoft Excel comma-separated.RFC4180 CSV - Comma-separated file that strictly.Default CSV - File that includes comma-separated.You can use the following delimited format types: Delimited The destination writes records asĭelimited data. Binary The stage writes binary data to a singleįield in the record. When using Avro compression, avoid configuring any otherĬompression properties in the stage. ![]() You can also compress data with an Avro-supported compressionĬodec. You can include the Avro schema in the output. Schema management to avoid losing track of the Improve performance, but requires the appropriate Optionally include the schema definition in the In the record header attribute, you can optionallyĬonfigure the stage to register the Avro schema The destination to look up the schema in theĬonfluent Schema Registry by the schema ID or The Confluent Schema Registry is a distributed Retrieve the schema from Confluent Schema Registry. Schema included in the avroSchema record header Of the following methods to specify the location of the Avro The processor can generate theįollowing types of data: Avro The stage writes records based on the Avro schema. The Data Generator processor generates recordsĭifferently based on the data format that you select. When you use the Data Generator processor, you specify the target field for the serialized record.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |