Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
1 result
Forked from an inaccessible project.
Name Last commit Last update
Assets
Logs
Packages
ProjectSettings
documentation
.gitignore
ProgrammingDMEnvironment.Editor.Plugins.csproj
ProgrammingDMEnvironment.Plugins.csproj
ProgrammingDMEnvironment.csproj
ProgrammingDMEnvironment.sln
README.md
TODO.md
app.config
packages.config
tutorial.md
README.md

For OpenDRIVE related scripts and tutorial, visit OpenDRIVE output.

Introduction

This project presents a programming interface and automated testing infrastructure for closed-loop autonomous vehicle simulation tests. For more information, please refer to our paper.

Setting up the project

A Windows machine is recommended, though this project will also run on an Ubuntu machine. Download and install Unity (tested to work on Unity 2019.2). Open the project as a Unity project. There are two scenes of interest:

  • Assets\Scenes\ProgrammingSampleUdacityTraining is the training scene, which can be used to generate training data.0
  • Assets\Scenes\ProgrammingSampleUdacityAutonomous is the autonomous scene where the autonomous vehicle can be controlled by an external program. Online events are recorded. The scenes have all relevant scripts attached to objects contained therein. An important object is the ScriptPlaceHolder. This contains the environment program that is loaded, which includes the set-up of the environment, test variables and actors.

Project Structure

  • The entire folder can be opened in Unity as a Unity project.
  • The Assets folder contains the scripts, scenes and relevant 3D resources.
  • Traffic System includes prefabs for road and traffic building.
  • Pedestrian System includes prefabs for building a pedestrian system.
  • The Udacity folder contains relevant scripts and prefabs from Udacity's self driving simulator project.

Usage Guide

Initial set-up.

Open the project in Unity (you will need to clone the entire project at https://gitlab.mpi-sws.org/mathur/ProgrammingDMEnvironment). Open the scene ProgrammingSampleUdacityAutonomous (located in Assets/Scenes). In the scene, click on the ScriptPlaceHolder object. Then, in the inspector, you will have many scripts that control the environment, but especially note:

  • Visualize: the support that generates the desired test/scenario
  • StraightRoad: specifies the desired road network and test The StraightRoad should have the box by the name checked.

This image shows the Unity editor, with key features highlighted in green. key features

If you want to run a different road network script, first remove the StraightRoad script by right-clicking and selecting the Remove Component option. To add a new script, simply drag it over to the inspector.

Some example scripts are provided in https://gitlab.mpi-sws.org/mathur/ProgrammingDMEnvironment/tree/master/Assets/Scripts/Program/Examples .

Creating Road Environments

Creating a simple straight road segments

The following example creates a straight road of variable number of lanes and length, then adds an autonomous vehicle with a collision monitor.

public class StraightRoad : EnvironmentProgramBaseClass
{
    //Add the ProDM vars here
    VarInterval<int> length;
    VarEnum<int> numLanes;

    //Environment variables
    VarInterval<float> lightIntensity;
    ProDMColor color;
    VarInterval<float> ambientIntensity;
    VarInterval<float> fogDensity;

    void Start() //needs to be called in the start because the setup in the RoadElementVisualization happens in the Awake()
    {
        
        fogDensity = new VarInterval<float>(val: 0, min: 0, max: 0.3f, desc: "FOG_DENSITY");
        lightIntensity = new VarInterval<float>(val: 1, min: 0, max: 10);
        color = new ProDMColor(new VarInterval<float>(val: 1f, min: 0f, max: 1f), new VarInterval<float>(val: 0.9f, min: 0f, max: 1f),
            new VarInterval<float>(val: 0.8f, min: 0f, max: 1f), 1f);
        ambientIntensity = new VarInterval<float>(val: 1.5f, min: 0f, max: 2.5f, desc: "AMBIENT_LIGHT_INTENSITY");

        //define the ProDM variables
        length = new VarInterval<int>(val: 100, changeable: true, min: 20, max: 4000);

        List<int> numLanesOptions = new List<int> { 2, 4, 6 };
        numLanes = new VarEnum<int>(numLanesOptions, 0);


        Test();
    }

    //create the visualization of the road network
    //NOTE: The code to make the road network should go here.
    public override void MakeView()
    {
        //create list of items to be visualized later
        List<object> list = new List<object>();

        //create a straight road segment
        RoadSegment cRoad1 = new RoadSegment(length, numLanes, nameof(cRoad1), Orientation.VERTICAL);
        list.Add(cRoad1);

        //place autonomous car on the road segment, and attach a collision monior
        AutonomousVehicle autonomouscar = new AutonomousVehicle(nameof(autonomouscar), cRoad1, 0.11f, LANE.PLUS_ONE, type: TestActorType.EGO_VEHICLE_AUTONOMOUS, monitor: "CollisionMonitor");
        list.Add(autonomouscar);

        //visualize the scene
        PlaceHolderForVisualizer.Visualize(list);

    }


    public override void MakeEnvironment()
    {
        //Environment settings
        LightSettings light = new LightSettings(intensity: lightIntensity, direction: Vector3.down, color: color, ambientIntensity: ambientIntensity);
        FogSettings fog = new FogSettings(density: fogDensity);

        EnvironmentSettings env = new EnvironmentSettings(light, fog);
        PlaceHolderForVisualizer.Visualize(env);
    }
}

The output is as: output straightRoad

We provide the option to add trees or residential zoning to different sides of the road and corners of the intersections. This line of code modifies the straight road to have trees on either side:

RoadSegment cRoad1 = new RoadSegment(length, numLanes, nameof(cRoad1), Orientation.VERTICAL, zone1: Zoning.TREES, zone2: Zoning.TREES);

Making more complicated networks

The following example creates a more complicated road network with intersections and AI vehicles.

public class ConnectedRoads : EnvironmentProgramBaseClass
{

    //Add the ProDM vars here
    VarInterval<int> length;
    VarEnum<int> numLanes;

    //Environment variables
    VarInterval<float> lightIntensity;
    ProDMColor color;
    VarInterval<float> ambientIntensity;
    VarInterval<float> fogDensity;

    void Start() //needs to be called in the start because the setup in the RoadElementVisualization happens in the Awake()
    {

        fogDensity = new VarInterval<float>(val: 0, min: 0, max: 0.3f, desc: "FOG_DENSITY");
        lightIntensity = new VarInterval<float>(val: 1, min: 0, max: 10);
        color = new ProDMColor(new VarInterval<float>(val: 1f, min: 0f, max: 1f), new VarInterval<float>(val: 0.9f, min: 0f, max: 1f),
            new VarInterval<float>(val: 0.8f, min: 0f, max: 1f), 1f);
        ambientIntensity = new VarInterval<float>(val: 1.5f, min: 0f, max: 2.5f, desc: "AMBIENT_LIGHT_INTENSITY");

        //define the ProDM variables
        length = new VarInterval<int>(val: 20, changeable: true, min: 10, max: 40);

        List<int> numLanesOptions = new List<int> { 2, 4, 6 };
        numLanes = new VarEnum<int>(numLanesOptions, 0);


        Test();
    }

    //create the visualization of the road network
    //NOTE: The code to make the road network should go here.
    public override void MakeView()
    {
        // make the T-intersection
        TIntersection int1 = new TIntersection(numLanes: numLanes, zone1: Zoning.TREES, zone2: Zoning.TREES, zone3: Zoning.RESIDENTIAL_ZONE);
        PlaceHolderForVisualizer.Visualize(int1);

        // make straight roads to connect to the Intersection
        RoadSegment cRoad1 = new RoadSegment(lengthOfRoad: length, numLanes: numLanes, zone1: Zoning.RESIDENTIAL_ZONE, zone2: Zoning.TREES);
        RoadSegment cRoad3 = new RoadSegment(lengthOfRoad: length, numLanes: numLanes, zone1: Zoning.RESIDENTIAL_ZONE, zone2: Zoning.TREES);
        PlaceHolderForVisualizer.Visualize(cRoad1);
        PlaceHolderForVisualizer.Visualize(cRoad3);

        // add straight roads to a list
        List<RoadElement> cRoadList = new List<RoadElement>
        {
            cRoad1,
            cRoad3
        };

        // make an encapsulating object of straight roads
        RoadElement cRoads = new RoadElement(cRoadList);
        PlaceHolderForVisualizer.Visualize(cRoads);

        // Connect the encapsulated object to T-Interesection
        // first specify the connection mapping
        Dictionary<PhysicalConnection, PhysicalConnection> connMap1 = new Dictionary<PhysicalConnection, PhysicalConnection>
        {
            { cRoad1._connections.AtIndex(0), int1._connections.AtIndex(0) },
            { cRoad3._connections.AtIndex(1), int1._connections.AtIndex(2) }
        };
        // then make the connection
        RoadElement finalconnection = cRoads.ConnectTo(r2: int1, connectionMap: connMap1);

        PlaceHolderForVisualizer.Visualize(finalconnection);

        PlaceHolderForVisualizer.connectRoadElements(connMap1, cRoads, finalconnection, int1, cRoads, true);

        //place autonomous vehicle and attach collision monitor
        AutonomousVehicle autonomouscar = new AutonomousVehicle(nameof(autonomouscar), cRoad1, 0.11f, LANE.PLUS_ONE, monitor: "CollisionMonitor", type: TestActorType.EGO_VEHICLE_AUTONOMOUS);
        PlaceHolderForVisualizer.Visualize(autonomouscar);


        //place AI vehicles in the scene
        AIVehicle car1 = new AIVehicle(nameof(car1), cRoad3, 0.15f, LANE.MINUS_ONE, Color.blue);
        PlaceHolderForVisualizer.Visualize(car1);

        AIVehicle car2 = new AIVehicle(nameof(car2), cRoad3, 0.36f, LANE.PLUS_ONE, Color.blue);
        PlaceHolderForVisualizer.Visualize(car2);

        AIVehicle car3 = new AIVehicle(nameof(car3), cRoad1, 0.3f, LANE.PLUS_ONE, Color.blue);
        PlaceHolderForVisualizer.Visualize(car3);

        AIVehicle car4 = new AIVehicle(nameof(car4), cRoad1, 0.7f, LANE.MINUS_ONE, Color.blue);
        PlaceHolderForVisualizer.Visualize(car4);

        AIVehicle car5 = new AIVehicle(nameof(car5), cRoad1, 0.5f, LANE.PLUS_ONE, Color.blue);
        PlaceHolderForVisualizer.Visualize(car5);

        AIVehicle car6 = new AIVehicle(nameof(car6), cRoad1, 0.5f, LANE.MINUS_ONE, Color.blue);
        PlaceHolderForVisualizer.Visualize(car6);

    }


    public override void MakeEnvironment()
    {
        //Environment settings
        LightSettings light = new LightSettings(intensity: lightIntensity, direction: Vector3.down, color: color, ambientIntensity: ambientIntensity);
        FogSettings fog = new FogSettings(density: fogDensity);

        EnvironmentSettings env = new EnvironmentSettings(light, fog);
        PlaceHolderForVisualizer.Visualize(env);
    }
}

The output is as: output connectedRoads

Further complicated scenes can be found in the Assets\Scripts\Programs\RoadSegmentation folder.

Environment Settings

Environment settings control the light intensity, fog, etc. of the simulation environment.

Adding Test Actors

Autonomous Vehicle

This test actor can be controlled manually (A-S-W-D keys on keyboard) or by connecting some autonomous controller. To use the training scene, ensure the type is TestActorType.EGO_VEHICLE_TRAINING and for the test scene, the type should be TestActorType.EGO_VEHICLE_AUTONOMOUS The collision monitor for the autonomous vehicle is located in https://gitlab.mpi-sws.org/mathur/ProgrammingDMEnvironment/tree/master/Assets/Scripts/Presenter/LiveChecks/Monitors

Pedestrians

Pedestrians currently come in one design. Behaviors are setup to allow the pedestrian to cross the road after a specified time, or when a specified actor of interest comes within a certain euclidian distance. These behaviors are located in https://gitlab.mpi-sws.org/mathur/ProgrammingDMEnvironment/tree/master/Assets/Scripts/Presenter/LiveChecks/Behaviors

The following line of code creates a pedestrian that crosses the road after a specified time:

Pedestrian pedestrian1 = new Pedestrian(nameof(pedestrian1), cRoad1, 0.2f, timeDelay: 5f, walkingSpeed: walkingSpeed, behavior: "PedestrianCrossingDelay");

The following line of code creates a pedestrian that crosses the road when a test actor (in this case, the autonomous car) comes within a certain Euclidian distance:

Pedestrian pedestrian1 = new Pedestrian(nameof(pedestrian1), cRoad1, normalizedDistance:distance, actorOfInterest:autonomouscar, euclideanDistance: euclideanDist, walkingSpeed:walkingSpeed, behavior: "PedestrianCrossingBehavior01");

AI Vehicles

AI vehicles are other traffic vehicles. They have their own built-in behaviors as a default, although more complex behaviors can be specified.

The model of AI vehicle can be altered by specifying in its instantiation:

AIVehicle car1 = new AIVehicle(nameof(car1), cRoad1, 0.15f, LANE.MINUS_ONE, Color.blue, model: "AICar5");

Generating Tests

Timed Test

Call the following function to run a test that lasts for 100 seconds (one iteration):

Test(timeCutOff: 100f);

Iterative Tests

The following program creates an iterative test. Explanations of all of the modes of iterative testing are further described below.

public class RandomSampling_100iterations : EnvironmentProgramBaseClass
{
    //Add the ProDM vars here
    VarInterval<int> cr_length;
    VarEnum<int> cr_numLanes;
    VarInterval<float> lightIntensity;
    ProDMColor color;
    VarInterval<float> ambientIntensity;
    VarInterval<float> fogDensity;

    // Parameterize using two ProDMFloats
    VarInterval<float> walkingSpeed;
    VarInterval<float> euclideanDist;

    void Start() //needs to be called in the start because the setup in the RoadElementVisualization happens in the Awake()
    {
		fogDensity = new VarInterval<float>(val: 0, min: 0, max: 0.3f, desc: "FOG_DENSITY");

		lightIntensity = new VarInterval<float>(val: 1, min: 0, max: 10);
		color = new ProDMColor(new VarInterval<float>(val: 1f, min: 0f, max: 1f), new VarInterval<float>(val: 0.9f, min: 0f, max: 1f),
			new VarInterval<float>(val: 0.8f, min: 0f, max: 1f), 1f);
		//PlaceHolderForVisualizer.Visualize(nameof(color), color);

		ambientIntensity = new VarInterval<float>(val: 1.5f, min: 0f, max: 2.5f, desc: "AMBIENT_LIGHT_INTENSITY");

		//define the ProDM variables
		cr_length = new VarInterval<int>(val: 100, changeable: true, min: 20, max: 4000);

		List<int> numLanesOptions = new List<int> { 2, 4 };

		cr_numLanes = new VarEnum<int>(numLanesOptions, 0);

		walkingSpeed = new VarInterval<float>(val: 4f, min: 2f, max: 10f);
		euclideanDist = new VarInterval<float>(val: 40f, min: 30f, max: 60f);

		List<VarInterval<float>> rangedFloats = new List<VarInterval<float>>();
		rangedFloats.Add(walkingSpeed);
		rangedFloats.Add(euclideanDist);

        TestIterative(rangedFloats:rangedFloats, iterations: 100, singleTestTime:15f, setVar: AutomatedVarSetMethod.RANDOM);

    }

	//create the visualization of the road network
	//NOTE: The code to make the road network should go here.
	public override void MakeView()
	{
		List<object> list = new List<object>();

		RoadSegment cRoad1 = new RoadSegment(cr_length, cr_numLanes, orientation: Orientation.VERTICAL);
		cRoad1._zone1 = Zoning.TREES;
		cRoad1._zone2 = Zoning.TREES;
		list.Add(cRoad1);


		AutonomousVehicle autonomouscar = new AutonomousVehicle(nameof(autonomouscar), cRoad1, 0.11f, LANE.PLUS_ONE, monitor: "CollisionMonitor", type: TestActorType.EGO_VEHICLE_AUTONOMOUS);
		list.Add(autonomouscar);

		Pedestrian pedestrian = new Pedestrian(nameof(pedestrian), cRoad1, 0.2f, actorOfInterest: autonomouscar, euclideanDistance: euclideanDist, walkingSpeed: walkingSpeed, behavior: "PedestrianCrossingBehavior01");
		list.Add(pedestrian);

		PlaceHolderForVisualizer.Visualize(list);
	}


	public override void MakeEnvironment()
	{
		LightSettings light = new LightSettings(intensity: lightIntensity, direction: Vector3.down, color: color, ambientIntensity: ambientIntensity);
		FogSettings fog = new FogSettings(density: fogDensity);

		EnvironmentSettings env = new EnvironmentSettings(light, fog);
		PlaceHolderForVisualizer.Visualize(env);
	}
}

Sampling

The following code will set up 100 test iterations that each last 15 seconds. rangedFloats contains the parameterized variables that will be tested.

TestIterative(rangedFloats: rangedFloats, iterations: 100, singleTestTime: 15f, setVar: AutomatedVarSetMethod.RANDOM);

Using AutomatedVarSetMethod.RANDOM sets each rangedFloat randomly, while using AutomatedVarSetMethod.LOW_DISCREPENCY will use the Halton sequence to select the rangedFloat values for each iteration;

Energy Maximization

Energy maximization will find teh maximum collision energy using a simulated annealing algorithm to test the rangedFloats. The sample size dictates how many numbers from the range of each rangedFloat value to test, and the repetitions says how many times the simulated annealing algorithm will run. In this case, 20 samples will be selected, with 5 rounds of simulated annealing performed, for a total of 100 iterations total, with each test lasting 15 seconds.

TestIterative(testVars: rangedFloats, repetitions: 5, sampleSize: 20, singleTestTime: 15f, actorMonitorName: "CollisionMonitor", setVar: AutomatedVarSetMethod.OPTIMIZE_ENERGY_HALTON);

Using AutomatedVarSetMethod.OPTIMIZE_ENERGY_RANDOM allows for random selection of each sample, while using AutomatedVarSetMethod.OPTIMIZE_ENERGY_HALTON will use the Halton sequence to select each sample;

A variant of the energy maximization uses the modes OPTIMIZE_ENERGY_HALTON_MAX and OPTIMIZE_ENERGY_RANDOM_MAX. This mode will first sample an initial n samples, then select the k highest energies and perform m repetitions of simulated annealing on each of the k samples.

For example, the following code will first take 80 samples, choose the top 5, and ask for 4 repetitions of each, for 100 total test iterations.

TestIterative(testVars: rangedFloats, repetitions: 4, sampleSize: 80, numTopEnergies: 5 singleTestTime: 15f, actorMonitorName: "CollisionMonitor", setVar: AutomatedVarSetMethod.OPTIMIZE_ENERGY_HALTON_MAX);

For all of these tests, you must specify the name of the monitor script that should be maximized. All of the above examples use the "CollisionMonitor" that is attached to the autonomous vehicle.

External packages used

This project uses: