📗 In this competition, you will use either supervised (imitation) learning or reinforcement learning techniques to train an autonomous vehicle to move on a continuous state space, similar to this project: Link, Link or Link. You will submit a policy network (the neural network that produces the actions based on the state of the environment) that outputs a deterministic Markov policy ("turn left", "turn right", "speed up" or "no action" given the state of the vehicle, including position, velocity and distances to walls or other vehicles observed by the sensors). The neural network should be fully connected with two hidden layers (ReLU activation, a maximum of 100 units in each layer) and input layer with \(4 + k\) units, and output layer with 4 units (softmax activation), where \(k\) is the number of sensors in front of the car, an odd integer between \(1\) and \(9\) (that is \(k\) must be \(1, 3, 5, 7, 9\)).
📗 Submit a policy network to compete in a random environment with other students. Game theoretic considerations can be made to modify your policy (through retraining with different training sets). You can optionally submit two policy networks and the maximum score from the two will be used in determining your ranking (the second vehicle can be designed to crash into other players and slow them down too).
📗 Your score will be the total distance traveled within a fixed number of frames, that is,
➩ Your score will be higher if your speed up more often.
➩ Your score will be higher if you crash fewer number of times.
📗 Details on computing the total distances:
➩ The cars are only allowed to move inside [0, 1] x [0, 1] region.
➩ There is no penalty for crashing with the walls or other vehicles except for the speed will decrease to 12.5% of the original speed.
➩ The action "speed up" will increase the speed by 2.5% per frame.
➩ The actions "turn left" and "turn right" will decrease the speed by 1.25% per frame, and change the angle (in radians) by 0.01.
➩ The speed is clamped between 0.001 and 0.1.
➩ The total distance is computed by summing up the lengths of the line segments connecting the positions between every two frames (it should be approximately proportional to the sum of lengths of the velocities too).
📗 You can use following demo to test policies or generate training sets (each line is ["action", "x", "y", "vx", "vy", "s1", "s2", ... "sk"], where "si" is distance measured by sensor i, from left (i = 1) to right (i = k)). The dark red lines represent how the sensors compute the distances for the vehicle you control. To collect data points, enter your network, the number of data points and number of frames to skip, then check the "Collect" checkbox to start.
Neural network to control the car: , Player icon:
Data set: Collect (max 1000) every frames.
Number of other cars (or networks to control them, separated by "====="):
Race track:
Game track: Create tracks
➩ You can create track that is a -sided regular polygon with width using this button: and click the "Restart" button.
➩ You can also try one of the F1 tracks: Link. You can create the track based on the following track centers with width using this button: and click the "Restart" button:
You can also manually control one vehicle (arrow keys or "wasd" to use actions "speed up" , "turn left", "no action", "turn right"). Not pressing any of the keys will perform "no action" too. Click anywhere inside the square (or check the "Collect" box) before you start.
Number of sensors: , Player icon:
Data set: Collect (max 1000) every frames.
Number of other cars (or networks to control them, separated by "====="):
Race track:
Game track: Create tracks
➩ You can create track that is a -sided regular polygon with width using this button: and click the "Restart" button.
➩ You can also try one of the F1 tracks: Link. You can create the track based on the following track centers with width using this button: and click the "Restart" button:
Your submission should contain (i) your player name (not necessarily your real name), (ii) your player icon (single emoji from this list: Link), (iii) your team (a random number between 0 and 1, rounded to four decimal places), (iv) your network weights, (v) [optional] your second network weights.