raw manhattan rows: 96088 raw brooklyn rows: 123813 clean manhattan rows: 7294 clean brooklyn rows: 42743 Min. 1st Qu. Median Mean 3rd Qu. Max. 1.005e+04 1.368e+06 4.000e+06 1.633e+07 9.575e+06 2.398e+09 25% 50% 75% 90% 95% 99% 1367750 4000000 9575000 22736850 50618132 266148692 manhattan iqr upper bound: 21885875 manhattan max sale price: 2397501899 number of sale price outliers: 756 sale_price gross_sqft land_sqft year_built res_units comm_units 1.00000000 0.49144986 0.16392585 0.02117231 0.11589681 0.20912401 total_units 0.16618138 linear model (log price) metrics on manhattan: r2: 0.1867558 rmse: 1.689562 mae: 1.260938 random forest (log price) metrics on manhattan: r2: 0.6978428 rmse: 1.031439 mae: 0.661946 random forest (log price) metrics on brooklyn (trained on manhattan): r2: 0.2442946 rmse: 1.305062 mae: 1.084852 manhattan classification subset rows: 6792 manhattan neighborhoods: 28 knn (manhattan) accuracy: 0.4774882 macro f1: 0.4174939 random forest classifier (manhattan) accuracy: 0.5841232 macro f1: 0.5328558 [1] 28 28 Reference Prediction ALPHABET CITY CHELSEA CHINATOWN CLINTON ALPHABET CITY 9 1 0 0 CHELSEA 0 23 2 1 CHINATOWN 0 0 10 0 CLINTON 0 2 0 17 EAST VILLAGE 2 2 2 0 FASHION 0 4 0 0 GRAMERCY 0 0 0 0 GREENWICH VILLAGE-CENTRAL 0 2 1 0 GREENWICH VILLAGE-WEST 1 3 0 1 HARLEM-CENTRAL 3 4 3 5 HARLEM-EAST 4 1 0 0 HARLEM-UPPER 1 1 0 1 KIPS BAY 0 2 0 0 LOWER EAST SIDE 1 3 2 0 MANHATTAN VALLEY 0 0 0 0 MIDTOWN CBD 0 1 1 0 MIDTOWN EAST 1 2 0 1 MIDTOWN WEST 0 3 1 1 MURRAY HILL 0 3 1 0 SOHO 0 2 1 0 TRIBECA 0 1 0 0 UPPER EAST SIDE (59-79) 1 3 0 2 UPPER EAST SIDE (79-96) 0 2 1 0 UPPER WEST SIDE (59-79) 1 1 0 1 UPPER WEST SIDE (79-96) 1 3 0 0 UPPER WEST SIDE (96-116) 0 1 0 1 WASHINGTON HEIGHTS LOWER 0 0 0 0 WASHINGTON HEIGHTS UPPER 0 0 0 1 Reference Prediction EAST VILLAGE FASHION GRAMERCY ALPHABET CITY 2 0 0 CHELSEA 1 1 0 CHINATOWN 0 1 0 CLINTON 1 0 0 EAST VILLAGE 33 0 0 FASHION 0 10 0 GRAMERCY 1 0 29 GREENWICH VILLAGE-CENTRAL 2 1 0 GREENWICH VILLAGE-WEST 2 0 1 HARLEM-CENTRAL 4 2 2 HARLEM-EAST 0 2 0 HARLEM-UPPER 0 0 0 KIPS BAY 0 0 0 LOWER EAST SIDE 0 1 0 MANHATTAN VALLEY 0 0 1 MIDTOWN CBD 0 3 2 MIDTOWN EAST 0 1 0 MIDTOWN WEST 0 2 0 MURRAY HILL 0 2 0 SOHO 0 0 0 TRIBECA 1 1 0 UPPER EAST SIDE (59-79) 1 0 1 UPPER EAST SIDE (79-96) 0 0 0 UPPER WEST SIDE (59-79) 0 1 0 UPPER WEST SIDE (79-96) 2 0 2 UPPER WEST SIDE (96-116) 1 0 0 WASHINGTON HEIGHTS LOWER 1 0 0 WASHINGTON HEIGHTS UPPER 0 0 0 Reference Prediction GREENWICH VILLAGE-CENTRAL GREENWICH VILLAGE-WEST ALPHABET CITY 0 0 CHELSEA 1 5 CHINATOWN 1 1 CLINTON 0 0 EAST VILLAGE 3 1 FASHION 0 0 GRAMERCY 0 0 GREENWICH VILLAGE-CENTRAL 14 3 GREENWICH VILLAGE-WEST 5 42 HARLEM-CENTRAL 1 4 HARLEM-EAST 1 1 HARLEM-UPPER 0 0 KIPS BAY 0 1 LOWER EAST SIDE 1 0 MANHATTAN VALLEY 0 0 MIDTOWN CBD 0 0 MIDTOWN EAST 1 1 MIDTOWN WEST 0 1 MURRAY HILL 0 0 SOHO 1 0 TRIBECA 0 0 UPPER EAST SIDE (59-79) 1 6 UPPER EAST SIDE (79-96) 1 6 UPPER WEST SIDE (59-79) 0 0 UPPER WEST SIDE (79-96) 2 3 UPPER WEST SIDE (96-116) 1 0 WASHINGTON HEIGHTS LOWER 0 0 WASHINGTON HEIGHTS UPPER 0 0 Reference Prediction HARLEM-CENTRAL HARLEM-EAST HARLEM-UPPER KIPS BAY ALPHABET CITY 2 0 0 0 CHELSEA 0 0 0 0 CHINATOWN 2 0 0 0 CLINTON 0 0 1 0 EAST VILLAGE 0 0 0 0 FASHION 0 0 0 0 GRAMERCY 0 0 0 0 GREENWICH VILLAGE-CENTRAL 0 0 0 0 GREENWICH VILLAGE-WEST 0 0 2 0 HARLEM-CENTRAL 158 15 21 0 HARLEM-EAST 9 35 2 0 HARLEM-UPPER 9 2 19 0 KIPS BAY 2 0 1 27 LOWER EAST SIDE 3 0 0 0 MANHATTAN VALLEY 1 4 0 0 MIDTOWN CBD 0 0 0 0 MIDTOWN EAST 2 0 0 2 MIDTOWN WEST 2 0 0 0 MURRAY HILL 2 1 0 0 SOHO 1 1 0 0 TRIBECA 0 0 0 0 UPPER EAST SIDE (59-79) 2 0 1 0 UPPER EAST SIDE (79-96) 2 4 0 1 UPPER WEST SIDE (59-79) 0 0 0 0 UPPER WEST SIDE (79-96) 0 0 0 0 UPPER WEST SIDE (96-116) 0 0 4 0 WASHINGTON HEIGHTS LOWER 7 3 1 0 WASHINGTON HEIGHTS UPPER 1 1 1 0 Reference Prediction LOWER EAST SIDE MANHATTAN VALLEY MIDTOWN CBD ALPHABET CITY 0 0 0 CHELSEA 4 0 0 CHINATOWN 0 1 0 CLINTON 0 0 0 EAST VILLAGE 3 0 0 FASHION 2 0 1 GRAMERCY 0 1 0 GREENWICH VILLAGE-CENTRAL 0 0 0 GREENWICH VILLAGE-WEST 1 0 0 HARLEM-CENTRAL 4 7 2 HARLEM-EAST 1 0 0 HARLEM-UPPER 0 0 0 KIPS BAY 0 0 0 LOWER EAST SIDE 47 2 0 MANHATTAN VALLEY 0 11 0 MIDTOWN CBD 0 0 12 MIDTOWN EAST 0 0 3 MIDTOWN WEST 0 2 1 MURRAY HILL 2 0 0 SOHO 0 0 1 TRIBECA 0 0 0 UPPER EAST SIDE (59-79) 1 0 2 UPPER EAST SIDE (79-96) 1 1 1 UPPER WEST SIDE (59-79) 0 0 2 UPPER WEST SIDE (79-96) 0 0 1 UPPER WEST SIDE (96-116) 0 1 0 WASHINGTON HEIGHTS LOWER 0 1 0 WASHINGTON HEIGHTS UPPER 0 1 0 Reference Prediction MIDTOWN EAST MIDTOWN WEST MURRAY HILL SOHO TRIBECA ALPHABET CITY 0 0 1 1 0 CHELSEA 3 1 2 1 0 CHINATOWN 0 1 2 0 0 CLINTON 0 0 1 2 0 EAST VILLAGE 1 0 0 2 0 FASHION 0 3 3 1 0 GRAMERCY 1 0 0 0 1 GREENWICH VILLAGE-CENTRAL 0 1 0 3 1 GREENWICH VILLAGE-WEST 1 0 1 3 1 HARLEM-CENTRAL 4 5 5 0 0 HARLEM-EAST 0 0 1 0 0 HARLEM-UPPER 0 1 2 0 0 KIPS BAY 0 0 0 0 0 LOWER EAST SIDE 0 2 1 2 1 MANHATTAN VALLEY 0 0 0 1 0 MIDTOWN CBD 0 6 0 1 0 MIDTOWN EAST 24 1 1 0 0 MIDTOWN WEST 0 177 0 2 0 MURRAY HILL 3 0 16 1 0 SOHO 1 1 1 16 1 TRIBECA 0 3 0 0 36 UPPER EAST SIDE (59-79) 1 0 2 2 2 UPPER EAST SIDE (79-96) 3 2 2 1 1 UPPER WEST SIDE (59-79) 2 0 0 0 0 UPPER WEST SIDE (79-96) 3 0 2 0 2 UPPER WEST SIDE (96-116) 1 0 0 0 0 WASHINGTON HEIGHTS LOWER 0 0 0 0 0 WASHINGTON HEIGHTS UPPER 0 0 0 0 0 Reference Prediction UPPER EAST SIDE (59-79) UPPER EAST SIDE (79-96) ALPHABET CITY 0 0 CHELSEA 1 3 CHINATOWN 1 4 CLINTON 1 1 EAST VILLAGE 2 1 FASHION 2 1 GRAMERCY 0 0 GREENWICH VILLAGE-CENTRAL 0 0 GREENWICH VILLAGE-WEST 2 7 HARLEM-CENTRAL 6 5 HARLEM-EAST 0 0 HARLEM-UPPER 1 2 KIPS BAY 0 0 LOWER EAST SIDE 1 1 MANHATTAN VALLEY 0 0 MIDTOWN CBD 2 2 MIDTOWN EAST 4 2 MIDTOWN WEST 1 0 MURRAY HILL 0 1 SOHO 2 2 TRIBECA 1 0 UPPER EAST SIDE (59-79) 47 14 UPPER EAST SIDE (79-96) 20 53 UPPER WEST SIDE (59-79) 2 1 UPPER WEST SIDE (79-96) 3 2 UPPER WEST SIDE (96-116) 0 3 WASHINGTON HEIGHTS LOWER 1 0 WASHINGTON HEIGHTS UPPER 0 1 Reference Prediction UPPER WEST SIDE (59-79) UPPER WEST SIDE (79-96) ALPHABET CITY 1 1 CHELSEA 0 2 CHINATOWN 0 0 CLINTON 0 1 EAST VILLAGE 1 1 FASHION 0 0 GRAMERCY 0 0 GREENWICH VILLAGE-CENTRAL 1 1 GREENWICH VILLAGE-WEST 0 1 HARLEM-CENTRAL 4 3 HARLEM-EAST 1 1 HARLEM-UPPER 0 1 KIPS BAY 0 0 LOWER EAST SIDE 0 0 MANHATTAN VALLEY 0 2 MIDTOWN CBD 0 0 MIDTOWN EAST 0 0 MIDTOWN WEST 3 1 MURRAY HILL 0 0 SOHO 0 0 TRIBECA 1 1 UPPER EAST SIDE (59-79) 4 2 UPPER EAST SIDE (79-96) 4 8 UPPER WEST SIDE (59-79) 46 2 UPPER WEST SIDE (79-96) 7 31 UPPER WEST SIDE (96-116) 1 0 WASHINGTON HEIGHTS LOWER 0 0 WASHINGTON HEIGHTS UPPER 0 1 Reference Prediction UPPER WEST SIDE (96-116) WASHINGTON HEIGHTS LOWER ALPHABET CITY 0 0 CHELSEA 1 0 CHINATOWN 0 0 CLINTON 0 0 EAST VILLAGE 0 0 FASHION 0 0 GRAMERCY 0 0 GREENWICH VILLAGE-CENTRAL 1 0 GREENWICH VILLAGE-WEST 0 0 HARLEM-CENTRAL 6 18 HARLEM-EAST 1 1 HARLEM-UPPER 1 5 KIPS BAY 1 0 LOWER EAST SIDE 1 1 MANHATTAN VALLEY 0 0 MIDTOWN CBD 0 0 MIDTOWN EAST 0 0 MIDTOWN WEST 0 0 MURRAY HILL 0 0 SOHO 0 0 TRIBECA 1 0 UPPER EAST SIDE (59-79) 1 0 UPPER EAST SIDE (79-96) 2 1 UPPER WEST SIDE (59-79) 1 0 UPPER WEST SIDE (79-96) 2 0 UPPER WEST SIDE (96-116) 16 0 WASHINGTON HEIGHTS LOWER 0 23 WASHINGTON HEIGHTS UPPER 1 2 Reference Prediction WASHINGTON HEIGHTS UPPER ALPHABET CITY 0 CHELSEA 0 CHINATOWN 0 CLINTON 0 EAST VILLAGE 0 FASHION 0 GRAMERCY 0 GREENWICH VILLAGE-CENTRAL 0 GREENWICH VILLAGE-WEST 0 HARLEM-CENTRAL 7 HARLEM-EAST 3 HARLEM-UPPER 4 KIPS BAY 0 LOWER EAST SIDE 0 MANHATTAN VALLEY 0 MIDTOWN CBD 0 MIDTOWN EAST 0 MIDTOWN WEST 0 MURRAY HILL 0 SOHO 1 TRIBECA 0 UPPER EAST SIDE (59-79) 0 UPPER EAST SIDE (79-96) 0 UPPER WEST SIDE (59-79) 0 UPPER WEST SIDE (79-96) 1 UPPER WEST SIDE (96-116) 0 WASHINGTON HEIGHTS LOWER 7 WASHINGTON HEIGHTS UPPER 5 multinomial logistic regression (manhattan) accuracy: 0.2517773 macro f1: 0.2957543 brooklyn classification subset rows: 42533 brooklyn neighborhoods: 56 contingency table dimensions (knn): 56 28 contingency table dimensions (random forest): 56 28 contingency table dimensions (logit): 56 28