csv - MySQL load ignores some records -


i have csv file 16.916 records. when load mysql, detects 15.945 records. thats mysql says:

records: 15945  deleted: 0  skipped: 0  warnings: 0 

can tell why mysql ignores records , how can fix this?

i load file using load function this:

load data local infile 'germany-filtered.csv' table point_of_interest fields terminated ','     enclosed '"' lines terminated '\n' ignore 1 lines (osm_id,lat,lng,access,addr_housename,addr_housenumber,addr_interpolation,admin_level,aerialway,aeroway,amenity,area,barrier,bicycle,brand,bridge,boundary,building,capital,construction,covered,culvert,cutting,denomination,disused,ele,embankment,foot,generator_source,harbour,highway,historic,horse,intermittent,junction,landuse,layer,leisure,ship_lock,man_made,military,motorcar,name,osm_natural,office,oneway,operator,place,poi,population,power,power_source,public_transport,railway,ref,religion,route,service,shop,sport,surface,toll,tourism,tower_type,tunnel,water,waterway,wetland,width,wood); 

thats database schema use:

create table point_of_interest (     `poi_id` int(10) unsigned not null auto_increment,     `lat` decimal(10, 8) default null,     `lng` decimal(11, 8) default null,     primary key  (`poi_id`),     key `lat` (`lat`),     key `lng` (`lng`),     osm_id bigint,     access text,     addr_housename text,     addr_housenumber text,     addr_interpolation text,     admin_level text,     aerialway text,     aeroway text,     amenity text,     area text,     barrier text,     bicycle text,     brand text,     bridge text,     boundary text,     building text,     capital text,     construction text,     covered text,     culvert text,     cutting text,     denomination text,     disused text,     ele text,     embankment text,     foot text,     generator_source text,     harbour text,     highway text,     historic text,     horse text,     intermittent text,     junction text,     landuse text,     layer text,     leisure text,     ship_lock text,     man_made text,     military text,     motorcar text,     name text,     osm_natural text,     office text,     oneway text,     operator text,     place text,     poi text,     population text,     power text,     power_source text,     public_transport text,     railway text,     ref text,     religion text,     route text,     service text,     shop text,     sport text,     surface text,     toll text,     tourism text,     tower_type text,     tunnel text,     water text,     waterway text,     wetland text,     width text,     wood text ) engine=innodb; 

update:

i checked first , last record both exist. records lot of empty values exist:

1503898236,10.5271308,52.7468051,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 

update 2:

those records found missing in database:

4228380062,9.9386752,53.6135468,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,dammwild,,,,,,,,,,,,,,,,,,,,attraction,,,,,,, 4228278589,9.9391503,53.5960304,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,kaninchen,,,,,,,,,,,,,,,,,,,,attraction,,,,,,, 4228278483,9.9396935,53.5960729,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,onager,,,,,,,,,,,,,,,,,,,,attraction,,,,,,, 4226772791,8.8394263,54.1354887,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,familienlagune perlebucht,,,,,,,,,,,,,,,,,,,,attraction,,,,,,, 

it seems of records osm_id starting 4 missing. thats strange.

try see if have duplicate id in file:

show file

# cat mycsv.csv 6991,10.4232704,49.4970160,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,bauernhaus aus seubersdorf,,,,,,,,,,,,,,,,,,,,attraction,,,,,,, 4228380062,9.9386752,53.6135468,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,dammwild,,,,,,,,,,,,,,,,,,,,attraction,,,,,,, 4228278589,9.9391503,53.5960304,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,kaninchen,,,,,,,,,,,,,,,,,,,,attraction,,,,,,, 4228278483,9.9396935,53.5960729,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,onager,,,,,,,,,,,,,,,,,,,,attraction,,,,,,, 4226772791,8.8394263,54.1354887,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,familienlagune perlebucht,,,,,,,,,,,,,,,,,,,,attraction,,,,,,, 4228278589,9.9391503,53.5960304,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,kaninchen,,,,,,,,,,,,,,,,,,,,attraction,,,,,,, 

count lines

# wc -l mycsv.csv 6 mycsv.csv 

remove duplicate ids , count again

# cut -d',' -f1 mycsv.csv | sort | uniq | wc -l 5 

Comments

Popular posts from this blog

sequelize.js - Sequelize group by with association includes id -

android - Robolectric "INTERNET permission is required" -

java - Android raising EPERM (Operation not permitted) when attempting to send UDP packet after network connection -