Assignment data preprocessing – ufo sighting data exploration

 I had attached the ufo_sightings_large.csv

  • In this assignment, you will investigate UFO data over the last century to gain some insight.
  • Please use all the techniques we have learned in the class to preprocesss/clean the datasetufo_sightings_large.csv
  • After the dataset is preprocessed, please split the dataset into training sets and test sets
  • Fit KNN to the training sets.
  • Print the score of KNN on the test sets


1. Import dataset “ufo_sightings_large.csv” in pandas (5 points)


2. Checking column types & Converting Column types (10 points)

Take a look at the UFO dataset’s column types using the dtypes attribute. Please convert the column types to the proper types. For example, the date column, which can be transformed into the datetime type. That will make our feature engineering efforts easier later on.


3. Dropping missing data (10 points)

Let’s remove some of the rows where certain columns have missing values.


4. Extracting numbers from strings (10 points)

The length_of_time column in the UFO dataset is a text field that has the number of minutes within the string. Here, you’ll extract that number from that text field using regular expressions.

In [ ]:


5. Identifying features for standardization (10 points)

In this section, you’ll investigate the variance of columns in the UFO dataset to determine which features should be standardized. You can log normlize the high variance column.


6. Encoding categorical variables (20 points)

There are couple of columns in the UFO dataset that need to be encoded before they can be modeled through scikit-learn. You’ll do that transformation here, using both binary and one-hot encoding methods.


7. Text vectorization (10 points)

Let’s transform the desc column in the UFO dataset into tf/idf vectors, since there’s likely something we can learn from this field.


8. Selecting the ideal dataset (10 points)

Let’s get rid of some of the unnecessary features.


9. Split the X and y using train_test_split, setting stratify = y (5 points)

In [9]:

X = ufo.drop(["type"],axis = 1)
y = ufo["type"].astype(str)


10. Fit knn to the training sets and print the score of knn on the test sets (5 points)

In [1]:

from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=5)
# Fit knn to the training sets, train_y)
# Print the score of knn on the test sets
print(knn.score(test_X, test_y))

Calculate Your Essay Price
(550 words)

Approximate price: $22

Calculate the price of your order

550 words
We'll send you the first draft for approval by September 11, 2018 at 10:52 AM
Total price:
The price is based on these factors:
Academic level
Number of pages
Basic features
  • Free title page and bibliography
  • Unlimited revisions
  • Plagiarism-free guarantee
  • Money-back guarantee
  • 24/7 support
On-demand options
  • Writer’s samples
  • Part-by-part delivery
  • Overnight delivery
  • Copies of used sources
  • Expert Proofreading
Paper format
  • 275 words per page
  • 12 pt Arial/Times New Roman
  • Double line spacing
  • Any citation style (APA, MLA, Chicago/Turabian, Harvard)

Our guarantees

Delivering a high-quality product at a reasonable price is not enough anymore.
That’s why we have developed 5 beneficial guarantees that will make your experience with our service enjoyable, easy, and safe.

Money-back guarantee

You have to be 100% sure of the quality of your product to give a money-back guarantee. This describes us perfectly. Make sure that this guarantee is totally transparent.

Read more

Zero-plagiarism guarantee

Each paper is composed from scratch, according to your instructions. It is then checked by our plagiarism-detection software. There is no gap where plagiarism could squeeze in.

Read more

Free-revision policy

Thanks to our free revisions, there is no way for you to be unsatisfied. We will work on your paper until you are completely happy with the result.

Read more

Privacy policy

Your email is safe, as we store it according to international data protection rules. Your bank details are secure, as we use only reliable payment systems.

Read more

Fair-cooperation guarantee

By sending us your money, you buy the service we provide. Check out our terms and conditions if you prefer business talks to be laid out in official language.

Read more

Enjoy 10% OFF today with the coupon code: best10