could anyone help me to my below query.
I have a duplicate records in a file .
I want to eliminate duplicate records in a file while loading into table ( my table is populated table), we cant use fastload
if the table is populated table, so how can we eliminate duplicate records in a file and which utility we can prefer to load only unique records.
Thanks in advance.....
Do you want to use unix/linux command? something like this
sort ... filename|uniq.....
or use awk.
You can see the arguement here :)
and you can see more in the forum.
Is your target table SET or MULTISET?
Is there a UPI or NUPI?
What are you going to do with those records, INSERT them into the target?
INSERTs into SET tables will never load duplicate rows, but might write them into the UV error table (controled by MARK/IGNORE DUPLICATE INSERT ROWS in DML LABEL).
Otherwise you might FastLoad to a staging table and then Insert/Select.
you can't do that with MultiLoad if the target table is MULTISET.
You got two options:
1. use Unix to remove duplicates before loading as Raja suggested
2. FastLoad to a staging table and then Insert/Select into your target table
To avoid duplicates, you have to use upsert with label "DO INSERT FOR MISSING UPDATE ROWS;" into your multiload script.
I have tried and it working for me. Please refer below for an example.
Below is my data file.