Description
Stock Price Analysis & Algorithmic Trading – Part 1 (100 pts)
This assignment is the first part of a 2-part assignment that focuses on providing you hands-on experience with building a real time stock price analysis and algorithmic trading model.
Part 1 would focus on data collection, storage and pre-processing to get your team ready to perform analysis and algorithmic trading on the stored data in Part 2. In this lab, you will work with your team to design and implement functions that would dynamically fetch stock data using the finance API.
Additionally, you’ll explore different ways to efficiently store and quickly retrieve the data from the database. 1. Database Installation and Configuration We will be working with MySQL to create our databases, store data into these tables using python scripts and query the data using SQL queries.
1. Installation Here’s a list of commands you need to follow to install MySQL and phpMyAdmin on your linux machine. 1.1. Install Apache web server. Apache is a web server software that will be used to host phpMyAdmin. Run the following command to install Apache: sudo apt install apache2
1.2. Enable Apache on firewall. The firewall may block access to Apache. To enable Apache on the firewall, run the following command: sudo ufw allow ‘Apache Full’
1.3. Install MySQL server. MySQL is a database management system that will be used to store data. Run the following command to install MySQL: sudo apt install mysql-server 1.4. MySQL security. By default, MySQL is installed with a weak password for the root user. To secure MySQL, run the following command to change the root password: sudo mysql_secure_installation [Note: This step
(1.4) will prompt you for a new password. In case it goes into a recursive loop which is a known issue for some installation
2 versions, follow the steps below to fix the issue and successfully set a password] https://www.digitalocean.com/community/tutorials/how-to-install-mysqlon-ubuntu-20-04#step-2-configuring-mysql
1.5. Install PHP. PHP is a programming language that is used to create dynamic web pages. Run the following command to install PHP: sudo apt install php
1.6. Install phpMyAdmin. phpMyAdmin is a web-based graphical user interface for MySQL. Run the following command to install phpMyAdmin: sudo apt install phpmyadmin 1.7. When prompted, choose Apache as the webserver to configure for phpMyAdmin. 1.8. Restart Apache. Finally, restart Apache to make the changes take effect: sudo service apache2 restart
1.9. Once you have completed these steps, you should be able to access phpMyAdmin by opening a web browser and navigating to the following address: http://localhost/phpmyadmin 1.10. You will be prompted to login with the root username and password that you set in step
5. Once you are logged in, you can create databases, tables, and users, and manage your MySQL data. 2. Install Required Libraries Run the following command to install the required python libraries. pip install mysql-connector-python pandas numpy yfinance
3. Reading Use the following resources to understand the MySQL setup and installation steps and learn about the functionalities they provide and their usage. Follow the documentation to install MySQL and PhpMyAdmin on your machine. 3.1. MySQL Documentation: Learn MySQL 3.2. SQL vs NoSQL: IBM Blogpost [Note: Though MySQL and phpMyAdmin is the preferred way to store data, your team is free to explore other options that you feel might be better alternatives. If you do use other alternatives, make sure you document the tools used and the rationale behind it] 2. Data Collection / Storage For this task, you will focus on creating the database tables, fetching stock data using the yfinance API the public dataset and writing a python script to populate the database tables from the dataset.
Use this yfinance to fetch real-time stock price data for a set of stocks. You can choose a specific set of companies or sectors that you’re interested in. The user should have an option to create a portfolio and define a list of stocks to be included in the portfolio. The script should accordingly fetch the stock price data for an input date range for the list of stocks in the portfolio. Write an additional script that lets the user manage their portfolio having operations such as adding a stock to the portfolio, removing a stock from the portfolio, displaying all their portfolios with the creation date and list of stocks in the portfolio.
Perform a validation check whether or not the given stock is valid and available in the yfinance API before actually adding it to the portfolio and display appropriate messages to the user such as: “added successfully ” or “invalid stock name”, etc. Resources: 1. Yahoo Finance API Documentation: yfinance GitHub 2. Why yfinance? : Link 3. Data Preprocessing Handle missing values in the collected data, considering strategies like forward/backward filling or interpolation. Transform the data into a suitable format for analysis, such as converting timestamps and calculating relevant metrics (e.g. daily returns, etc). Define the metrics which you feel would help you in the analysis and prediction of stock prices in Part-2 of the assignment.
Resources: 1. Handling Missing Data with Pandas: Dealing with Missing Data in Pandas 4. Team Discussions Your team is expected to meet in-person / virtually each day of the week and discuss the assignment progress & next steps. Document and compile minutes of all meetings in a separate file called ‘meeting_notes_A3_P1_.pdf’ Submission Make one submission per team. You will see a group with your team name created on the blackboard. Use the group to make submissions for group assignments. Each team must submit all the code files for the working solution, a readme document containing information for running the code in pdf format and a document that outlines the minutes of all team meetings in pdf format.
Provide a video per team which demonstrates the entire working solution and explains how the data tables were loaded, demonstrate query results and talk about the design decisions made along with reasoning for the same. Also include details about how your team preprocessed the data. Please include the team name and the name of all three team members in the video. There will be a 50% penalty for all late submissions.