Module: ml

Data primitives and data loaders

Warning

pykitPIV’s PIV images, and the associated targets, are generated assuming that the origin of the coordinate system is the lower-left corner (equivalent to setting origin='lower' in plt.imshow()). The bottom boundary of a PIV image corresponds to [0,:] rows from the raw numpy arrays that store, e.g., the image intensities or the velocity components. The top boundary of a PIV image corresponds to [-1,:] rows from the raw numpy arrays.

Whenever the PIV images need to be interpreted with the origin in the upper-left corner (equivalent to setting origin='upper' in plt.imshow()), the \(v\)-component of velocity has to be multiplied by \(-1\). This is the case when using the generated pykitPIV dataset as raw numpy arrays, such as in an input/output to a convolutional neural network (CNN). A CNN processes arrays assuming that the top boundary is [0,:] and the bottom boundary is [-1,:]. With this flip, the \(v\)-component of velocity has to swap sign, such that whatever was a positive \(v\)-component (from the old bottom to top) now is a negative \(v\)-component (from the new top to bottom).

This preserves the effect (divergence or convergence) that the \(v\)-component of velocity has on particles.

../_images/pykitPIV-dataloader-warning.svg

PyTorch-based

For more information on PyTorch data primitives visit this PyTorch documentation website.

class pykitPIV.ml.PIVDatasetPyTorch(dataset, transform=None)

Loads and stores the pykitPIV-generated dataset for PyTorch.

Note

The image intensities have to be indexed by "I" and the targets have to be indexed by "targets" within the dataset dictionary.

This is a subclass of torch.utils.data.Dataset.

Example:

from pykitPIV import PIVDatasetPyTorch

# Specify the path to the saved dataset:
path = 'docs/data/pykitPIV-dataset-10-PIV-pairs-256-by-256.h5'

# Load and store the dataset:
PIV_data = PIVDatasetPyTorch(dataset=path)
Parameters:

  • dataset – str specifying the path to the saved dataset. str specifying the path to the saved dataset. It can also be directly passed as a dict defining the pykitPIV dataset.

  • transform – (optional) torchvision.transform specifying vision transformations to augment the training dataset.

TensorFlow-based or Keras-based

For more information on TensorFlow data primitives visit this TensorFlow documentation website.

class pykitPIV.ml.PIVDatasetTF(dataset, transform=None)

Loads and stores the pykitPIV-generated dataset for TensorFlow or Keras.

Note

The image intensities have to be indexed by "I" and the targets have to be indexed by "targets" within the dataset dictionary.

This is a subclass of tf.keras.utils.PyDataset.

Example:

from pykitPIV import PIVDatasetTF

# Specify the path to the saved dataset:
path = 'docs/data/pykitPIV-dataset-10-PIV-pairs-256-by-256.h5'

# Load and store the dataset:
PIV_data = PIVDatasetTF(dataset=path)
Parameters:

  • dataset – str specifying the path to the saved dataset. str specifying the path to the saved dataset. It can also be directly passed as a dict defining the pykitPIV dataset.

  • transform – (optional) callable specifying vision transformations to augment the training dataset.

Variational and generative approaches

class pykitPIV.ml.PIVCVAE(*args, **kwargs)

Provides a convolutional variational autoencoder (CVAE) for generating new velocity fields, displacement fields, or image intensities samples that match in distribution those coming from an experiment. This approach can be used to extend the training data for transfer learning and can help adapt a machine learning model to the changing experimental conditions.

For more information on building convolutional VAEs, the user is referred to this TensorFlow’s CVAE tutorial.

The general workflow for augmenting training datasets with samples that fall in the distribution of a given experiment is illustrated below:

../_images/PIVCVAE.svg

This is a subclass of tensorflow.keras.Model.

Example:

from pykitPIV import PIVCVAE

# Specify the shape of the images at the input of CVAE:
input_shape = (256, 256, 2)

# Specify the latent dimension (typically mean + log-variance):
latent_dimension = 2

# Instantiate a CVAE model:
model = PIVCVAE(input_shape=input_shape,
                latent_dimension=latent_dimension)

Note

Note that TensorFlow’s convention for image size for convolutions is different from that of PyTorch. While PyTorch accepts \((N, C, H, W)\), TensorFlow uses the convention \((N, H, W, C)\) with the channel being the last dimension. To prepare pykitPIV’s images for TensorFlow’s convolutional layers, you can simply run the following transpose:

PIV_images = np.transpose(PIV_images, (0, 2, 3, 1))
Parameters:

  • input_shape – tuple of int specifying the path shape of the input image, or image pair. Typically, this will be (H, W, 1) for a single scalar quantity or (H, W, 2) for 2D vector quantities. For the moment, height and width should be divisible by 4 due to the architecture used.

  • latent_dimension – (optional) int specifying the latent dimension of the CVAE.

pykitPIV.ml.PIVCVAE.sample(self, eps=None)

Draws an image sample from decoding the latent space using the random vector, \(\varepsilon\).

Parameters:

eps – (optional) tensorflow.Tensor specifying the random noise, \(\varepsilon\), for random new sample generation. If set to None, a tensor of 100 random noise values will be generated from the normal distribution.

Returns:

  • sample - tensorflow.Tensor specifying the new image sample.

pykitPIV.ml.PIVCVAE.encode(self, x)

Encodes an image sample down to the mean, \(\mu\), and the log-variance, \(\ln(\sigma^2)\).

Parameters:

x – (optional) tensorflow.Tensor specifying the input image, preprocessed as necessary.

Returns:

  • mean - tensorflow.Tensor specifying the mean of the distribution.

  • logvar - tensorflow.Tensor specifying the log-variance, \(\ln(\sigma^2)\), of the distribution.

pykitPIV.ml.PIVCVAE.reparameterize(self, mean, logvar)

Computes the reparameterization trick to generate a sample \(z\):

\[z = \mu + \sigma \times \varepsilon\]

where \(\mu\) is the mean, \(\sigma\) is the standard deviation of the distribution, and \(\varepsilon\) is the random noise. Note that

\[\sigma = \sqrt{\exp(\ln(\sigma^2))} = \exp \big( \frac{1}{2} \cdot \ln(\sigma^2) \big)\]

and therefore \(z\) is computed in this function as:

\[z = \mu + \exp \big( \frac{1}{2} \cdot \ln(\sigma^2) \big) \times \varepsilon\]
Parameters:

  • mean – tensorflow.Tensor specifying the mean of the distribution.

  • logvar – tensorflow.Tensor specifying the log-variance, \(\ln(\sigma^2)\), of the distribution.

Returns:

  • z - tensorflow.Tensor specifying the latent variable, \(z\).

pykitPIV.ml.PIVCVAE.decode(self, z, apply_sigmoid=False)

Decodes a sample of the latent space into logits or probabilities.

Parameters:

  • z – tensorflow.Tensor specifying the latent variable.

  • apply_sigmoid – (optional) bool specifying whether the sigmoid should be applied to the logits.

Returns:

  • logits - tensorflow.Tensor specifying the logits if apply_sigmoid=False or probabilities if apply_sigmoid=True.

Reinforcement learning environments

Gymnasium-based

Below, you will find reinforcement learning (RL) environments that are subclasses of gymnasium.Env - a class coming from OpenAI’s Gymnasium library that provides the basic structure for any RL environment.

class pykitPIV.ml.PIVEnv(interrogation_window_size, interrogation_window_size_buffer, cues_function=None, particle_spec=None, motion_spec=None, image_spec=None, flowfield_spec=None, user_flowfield=None, inference_model=None, random_seed=None)

Provides a Gymnasium-based virtual PIV environment for a reinforcement learning (RL) agent.

The environment simulates a 2D section in a wind tunnel of a user-specified size, with synthetic or user-specified static velocity field, \(\vec{V} = [u, v]\), and provides synthetic PIV recordings under a (usually much smaller) interrogation window.

The flow target for the RL agent is the displacement field, \(d\vec{\mathbf{s}} = [dx, dy] = [u \Delta t, v \Delta t]\). This flow target is the basis for computing sensory cues and rewards.

The overall mechanics of this class is visualized below:

../_images/PIVEnv.svg

We refer to \(H_{\text{wt}}\) as the height and \(W_{\text{wt}}\) as the width of the virtual wind tunnel, and to \(H_{\text{i}}\) as the height and \(W_{\text{i}}\) as the width of the interrogation window.

The RL agent interacting with this environment is free to locate an interrogation window within the larger flow field satisfying certain condition modeled by the reward function (see the parameter reward_function in the PIVEnv.step() method). The agent moves smoothly, i.e., pixel-by-pixel from one interrogation window to the next, always performing one of the five actions:

  • 0: Move up by one pixel

  • 1: Move right by one pixel

  • 2: Move down by one pixel

  • 3: Move left by one pixel

  • 4: Stay

Future functionality will include temporally-evolving flow fields.

This is a subclass of gymnasium.Env.

Example:

from pykitPIV.ml import PIVEnv

# Prepare specs for pykitPIV parameters:
particle_spec = {'diameters': (1, 1),
                 'distances': (2, 2),
                 'densities': (0.2, 0.2),
                 'diameter_std': 1,
                 'seeding_mode': 'random'}

flowfield_spec = {'size': (500, 1000),
                  'time_separation': 5,
                  'flowfield_type': 'random smooth',
                  'gaussian_filters': (30, 30),
                  'n_gaussian_filter_iter': 5,
                  'displacement': (2, 2)}

motion_spec = {'n_steps': 10,
               'particle_loss': (0, 2),
               'particle_gain': (0, 2)}

image_spec = {'exposures': (0.5, 0.9),
              'maximum_intensity': 2**16-1,
              'laser_beam_thickness': 1,
              'laser_over_exposure': 1,
              'laser_beam_shape': 0.95,
              'alpha': 1/8,
              'clip_intensities': True,
              'normalize_intensities': False}

# Define a custom cues function that returns an array of three cues:
def cues_function(displacement_field_tensor):

    mean_displacement = np.mean(displacement_field_tensor)
    max_displacement = np.max(displacement_field_tensor)
    min_displacement = np.min(displacement_field_tensor)

    cues = np.array([[mean_displacement, max_displacement, min_displacement]])

    return cues

# Initialize the Gymnasium environment:
env = PIVEnv(interrogation_window_size=(100, 200),
             interrogation_window_size_buffer=10,
             cues_function=cues_function,
             particle_spec=particle_spec,
             motion_spec=motion_spec,
             image_spec=image_spec,
             flowfield_spec=flowfield_spec,
             user_flowfield=None,
             inference_model=None,
             random_seed=100)

Below is an example of importing a user-defined flow field and using it as the flow in the virtual wind tunnel. This can be done by uploading an external velocity field tensor to a FlowField class object. Below, we demonstrate this on a velocity field generated using the FlowField class, but the methodology is such that the user can replace the velocity_field tensor with a custom tensor that can be upladed externally, say, from a Johns Hopkins Turbulence Database (JHTD).

from pykitPIV import FlowField

# Initialize an object of the FlowField class that will store the user-specified flow field:
user_flowfield = FlowField(1,
                           size=(500, 1000),
                           size_buffer=0,
                           time_separation=1)

# Create a dummy velocity field:
user_flowfield.generate_random_velocity_field(displacement=(10, 10),
                                         gaussian_filters=(30, 30),
                                         n_gaussian_filter_iter=6)

velocity_field = flowfield.velocity_field

# Upload a user-specified velocity field tensor:
user_flowfield.upload_velocity_field(velocity_field)

# Initialize the Gymnasium environment:
env = PIVEnv(interrogation_window_size=(100, 200),
             interrogation_window_size_buffer=10,
             cues_function=cues_function,
             particle_spec=particle_spec,
             motion_spec=motion_spec,
             image_spec=image_spec,
             flowfield_spec=None,
             user_flowfield=user_flowfield,
             inference_model=None,
             random_seed=100)

Note that at any stage of training the RL agent, the time separation between two PIV image frames can be updated using the time_separation attribute:

env.time_separation = 2
Parameters:

  • interrogation_window_size – tuple of two int elements specifying the size of the interrogation window in pixels \([\text{px}]\). The first number is the window height, \(H_{\text{i}}\), the second number is the window width, \(W_{\text{i}}\).

  • interrogation_window_size_buffer – int specifying the buffer, \(b\), in pixels \([\text{px}]\) to add to the interrogation window size in the width and height direction. This number should be approximately equal to the maximum displacement that particles are subject to in order to allow new particles to arrive into the image area and prevent spurious disappearance of particles near image boundaries.

  • cues_function – function specifying the computation of cues that the RL agent senses based on the current displacement field reconstructed from PIV image pairs. The cues are the input parameters to the Q-network, hence the goal of the RL is to learn the mapping: \(\text{cues} \rightarrow \text{actions}\). This function has to take as an input the predicted displacement field tensor and return a numpy array of shape (1, N) of \(N\) cues computed from the predicted displacement field tensor. If set to None, the cues will be the whole displacement field tensor.

  • particle_spec – (optional) dict or particle.ParticleSpec object containing specifications for constructing an instance of Particle class.

  • motion_spec – (optional) dict or motion.MotionSpec object containing specifications for constructing an instance of Motion class.

  • image_spec – (optional) dict or image.ImageSpec object containing specifications for constructing an instance of Image class.

  • flowfield_spec – (optional) dict or flowfield.FlowFieldSpec object containing specifications for constructing an instance of FlowField class. If not specified, the user has to provide their own flow field through the user_flowfield parameter.

  • user_flowfield – (optional) object of pykitPIV.flowfield.FlowField class specifying the user-uploaded flow field for the entire virtual wind tunnel. If not specified, the user has to provide a flow field specification through the flowfield_spec parameter to create a synthetic pykitPIV-generated flow field. Future functionality will include temporally-evolving flow fields.

  • inference_model – (optional) object of a custom, user-defined class specifying the inference model for predicting flow targets from PIV images. It can be a CNN-based or a WIDIM-based model. If set to None, PIV images are not recorded and inference from them is not done, instead the true flow target within the interrogation window is returned. The inference model has to have a method inference_model.inference() that returns the predicted flow targets tensor of size \((1, 2, H_{\text{i}}+2b, W_{\text{i}}+2b)\). The inference_model.inference() method must be able to take as an input the raw PIV images and pre-process them as needed.

  • random_seed – (optional) int specifying the random seed for random number generation in numpy. If specified, all operations are reproducible.

Attributes:

  • flowfield - (read-only) as per user input.

  • flowfield_type - (read-only) as per user input.

  • flowfield_size - (read-only) as per user input.

  • time_separation - (can be re-set) as per user input.

  • admissible_observation_space - (read-only) tuple specifying the admissible observation space along height and width.

  • camera_position - (read-only) tuple specifying the camera position.

  • action_to_direction - (read-only) dict specifying how action numbers translate to directions in the 2D virtual environment.

  • action_to_verbose_direction - (read-only) dict specifying how action numbers translate to verbose direction strings in the 2D virtual environment.

  • n_actions - (read-only) int specifying the total number of actions possible in this environment.

  • prediction_tensor - (read-only) numpy.ndarray specifying the predicted displacement field tensor.

  • targets_tensor - (read-only) numpy.ndarray specifying the ground truth displacement field tensor.

pykitPIV.ml.PIVEnv.reset(self, imposed_camera_position=None, regenerate_flowfield=False)

Resets the environment to a random or user-imposed initial state and, optionally, also re-generates the flow field.

Example:

import numpy as np

# Once the environment has been initialized:
env = PIVEnv(...)

# We reset the environment to create its random initial state:
initial_camera_position, cues = env.reset()

# Optionally, we can impose an initial state:
initial_camera_position, cues = env.reset(imposed_camera_position=np.array([10, 50]))
Parameters:

  • imposed_camera_position – (optional) numpy.ndarray of two elements specifying the initial camera position in pixels \([\text{px}]\). This defines the bottom-left corner of the interrogation window. Example can be numpy.array([10, 50]) which positions the camera at the location \(10 \text{px}\) along the height dimension and \(50 \text{px}\) along the width dimension. If not specified, a random camera position is selected through env.observation_space.sample().

  • regenerate_flowfield – (optional) bool specifying whether the larger flow field should be regenerated at reset.

Returns:

  • camera_position - numpy.ndarray of two elements specifying the initial camera position in pixels \([\text{px}]\).

  • cues - numpy.ndarray specifying the initial cues that the RL agent will later be sensing.

pykitPIV.ml.PIVEnv.step(self, action, reward_function, reward_transformation, magnify_step=1, verbose=False)

Makes one step in the environment which moves the camera to a new position, and computes the associated reward for taking that step. The reward is computed based on the PIV images seen at that position, converted to a continuous displacement field recovered by an inference model (either CNN-based or WIDIM-based).

Example:

from pykitPIV.ml import Rewards

# Once the environment has been initialized:
env = PIVEnv(...)

# We reset the environment to create its initial state:
initial_camera_position, initial_cues = env.reset()

# We create a reward function by polling from one of the pykitPIV.Rewards methods.
# Here, we use the reward based on the Q-criterion:
rewards = Rewards(verbose=True)
reward_function = rewards.q_criterion

# We also define a function that will transform and reduce the Q-criterion
# to provide one reward value:
def reward_transformation(Q):
    Q = np.max(Q.clip(min=0))
    return Q

# Now we can take a step in the environment by selecting one of the five actions:
camera_position, cues, reward = env.step(action=4,
                                         reward_function=reward_function,
                                         reward_transformation=reward_transformation,
                                         verbose=True)
Parameters:

  • action – int specifying the action to be taken at the current step in the environment.

  • reward_function – function specifying the dynamics of the reward construction as a function of predicted and/or true flow targets. It can be one of the rewards functions from the pykitPIV.Rewards class.

  • reward_transformation – function specifying an arbitrary transformation of the reward function and an arbitrary compression of the reward function to a single value.

  • magnify_step – (optional) int specifying the factor that multiplies each unit step in the environment.

  • verbose – (optional) bool specifying if the verbose print statements should be displayed.

Returns:

  • camera_position - numpy.ndarray of two elements specifying the new camera position in pixels \([\text{px}]\) after taking this step.

  • cues - numpy.ndarray specifying the cues that the RL agent senses after taking this step.

  • reward - float specifying the reward received after taking this step.

pykitPIV.ml.PIVEnv.render(self, quantity=None, camera_position=None, wind_tunnel_only=False, c='white', s=10, lw=2, xlabel=None, ylabel=None, xticks=True, yticks=True, cmap='viridis', normalize_cbars=False, add_quiver=False, quiver_step=10, quiver_color='k', add_streamplot=False, streamplot_density=1, streamplot_color='k', streamplot_linewidth=1, figsize=(10, 5), dpi=300, filename=None)

Renders the virtual wind tunnel with the current interrogation window.

Example:

# Once the environment has been initialized:
env = PIVEnv(...)

# We can sample an observation which is the camera position:
camera_position = env.observation_space.sample()

# And we can render the virtual wind tunnel with the current interrogation window:
env.render(quantity,
           camera_position,
           wind_tunnel_only=False,
           c='white',
           s=20,
           lw=1,
           xlabel=None,
           ylabel=None,
           xticks=True,
           yticks=True,
           cmap='viridis',
           normalize_cbars=False,
           add_quiver=False,
           quiver_step=10,
           quiver_color='k',
           add_streamplot=False,
           streamplot_density=1,
           streamplot_color='k',
           streamplot_linewidth=1,
           figsize=(10, 5),
           dpi=300,
           filename=None)

One example rendering of the virtual wind tunnel can look like this:

../_images/ml_PIVEnv_render.png
Parameters:

  • quantity – (optional) numpy.ndarray specifying the quantity to plot within the virtual wind tunnel section. It should have size \((H_{\text{wt}}, W_{\text{wt}})\). If set to None, displacement field magnitude, \(|d\vec{\mathbf{s}}|\), is plotted.

  • camera_position – (optional) numpy.ndarray specifying the camera position in pixels \([\text{px}]\). This defines the bottom-left corner of the interrogation window. Example can be numpy.array([10,50]) which positions camera at the location \(10 \text{px}\) along the height dimension and \(50 \text{px}\) along the width dimension.

  • wind_tunnel_only – (optional) bool specifying if only the wind tunnel should be plotted.

  • c – (optional) str specifying the color for the interrogation window outline.

  • s – (optional) int or float specifying the size of the dot that represents the camera position.

  • lw – (optional) int or float specifying the line width for the interrogation window outline.

  • xlabel – (optional) str specifying \(x\)-label.

  • ylabel – (optional) str specifying \(y\)-label.

  • xticks – (optional) bool specifying if ticks along the \(x\)-axis should be plotted.

  • yticks – (optional) bool specifying if ticks along the \(y\)-axis should be plotted.

  • cmap – (optional) str or an object of matplotlib.colors.ListedColormap specifying the color map to use.

  • normalize_cbars – (optional) bool specifying if the colorbar for the interrogation window should be normalized to the colorbar for the entire wind tunnel.

  • add_quiver – (optional) bool specifying if vector field should be plotted on top of the scalar magnitude field.

  • quiver_step – (optional) int specifying the step on the pixel grid to attach a vector to. The higher this number is, the less dense the vector field is.

  • quiver_color – (optional) str specifying the color of velocity vectors.

  • add_streamplot – (optional) bool specifying if streamlines should be plotted on top of the scalar magnitude field.

  • streamplot_density – (optional) float or int specifying the streamplot density.

  • streamplot_color – (optional) str specifying the streamlines color.

  • streamplot_linewidth – (optional) int or float specifying the line width for the streamplot.

  • figsize – (optional) tuple of two numerical elements specifying the figure size as per matplotlib.pyplot.

  • dpi – (optional) int specifying the dpi for the image.

  • filename – (optional) str specifying the path and filename to save an image. If set to None, the image will not be saved.

Returns:

  • plt - matplotlib.pyplot image handle.

pykitPIV.ml.PIVEnv.record_particles(self, camera_position)

Creates virtual PIV recordings based on the current interrogation window.

Example:

# Once the environment has been initialized:
env = PIVEnv(...)

# We can sample an observation which is the camera position:
camera_position = env.observation_space.sample()

# We can now create a virtual PIV recording at that camera position:
image_obj = env.record_particles(camera_position)
Parameters:

camera_position – numpy.ndarray specifying the camera position in pixels \([\text{px}]\). This defines the bottom-left corner of the interrogation window. Example can be numpy.array([10,50]) which positions camera at the location \(10 \text{px}\) along the height dimension and \(50 \text{px}\) along the width dimension.

Returns:

  • image - pykitPIV.image.Image object specifying the generated PIV image pairs.

pykitPIV.ml.PIVEnv.make_inference(self, image_obj)

Makes inference of the displacement field under the interrogation window based on the recorded PIV images.

Example:

# Once the environment has been initialized:
env = PIVEnv(...)

# And once the PIV images have been recorded:
camera_position = env.observation_space.sample()
image_obj = env.record_particles(camera_position)

# We can now make inference and predict the displacement field:
targets_tensor, prediction_tensor = env.make_inference(image_obj)
Parameters:

image_obj – object of pykitPIV.image.Image class specifying the generated PIV image pairs.

Returns:

  • targets_tensor - numpy.ndarray specifying the tensor of true flow targets.

  • prediction_tensor - numpy.ndarray specifying the tensor of predicted flow targets.

TF-Agents-based

Note

The future version will include environments that can be directly used with TF-Agents.

Reinforcement learning agents

Single deep Q-network training

class pykitPIV.ml.CameraAgentSingleDQN(env, q_network, learning_rate=0.0001, optimizer='RMSprop', discount_factor=0.95, random_seed=None)

Creates a reinforcement learning (RL) agent that operates a virtual camera in a PIV experimental setting and provides a training loop for Q-learning. Here, we use Q-learning with a single deep neural network (DNN) model.

The goal of the RL agent is to learn the policy, which is mapping function, \(\pi\), from \(\text{cues} \rightarrow \text{actions}\) such that:

\[\text{action} = \pi( \text{cues} )\]

and where \(\pi\) is the trained DNN model.

Example:

from pykitPIV.ml import PIVEnv
from pykitPIV.ml import CameraAgentSingleDQN
import tensorflow as tf

# Initialize the environment:
env = PIVEnv(...)

# Example single deep Q-network model:
class QNetwork(tf.keras.Model):

    def __init__(self, n_actions, kernel_initializer):

        super(QNetwork, self).__init__()

        self.dense1 = tf.keras.layers.Dense(env.n_cues, activation='relu', kernel_initializer=kernel_initializer)
        self.dense2 = tf.keras.layers.Dense(size_of_hidden_unit, activation='relu', kernel_initializer=kernel_initializer)
        self.dense3 = tf.keras.layers.Dense(size_of_hidden_unit, activation='relu', kernel_initializer=kernel_initializer)
        self.output_layer = tf.keras.layers.Dense(n_actions, activation='relu', kernel_initializer=kernel_initializer)

    def call(self, state):

        x = self.dense1(state)
        x = self.dense2(x)
        x = self.dense3(x)

        return self.output_layer(x)

# Initialize the camera agent for single deep Q-network training:
ca = CameraAgentSingleDQN(env=env,
                          q_network=QNetwork(env.n_actions, tf.keras.initializers.RandomUniform),
                          learning_rate=0.0001,
                          optimizer='Adam',
                          discount_factor=0.95)
Parameters:

  • env – object of a custom environment class that is a subclass of gym.Env specifying the virtual environment. This can for instance be an object of the pykitPIV.ml.PIVEnv class.

  • q_network – tf.keras.Model specifying the single deep neural network that will be trained for Q-learning.

  • learning_rate – (optional) float specifying the initial learning rate. The learning rate can be updated on the fly by passing a new value to the train() function.

  • optimizer – (optional) str specifying the gradient descent optimizer to use. It can be 'Adam' or 'RMSprop'.

  • discount_factor – (optional) float specifying the discount factor, \(\gamma\).

  • random_seed – (optional) int specifying the random seed for random number generation in numpy. If specified, all operations are reproducible.

pykitPIV.ml.CameraAgentSingleDQN.choose_action(self, cues, epsilon)

Defines an \(\varepsilon\)-greedy choice of the next best action to select.

If the probability is less than \(\varepsilon\), action is selected at random.

If the probability is higher than or equal to \(\varepsilon\), action is selected based on the cues that are the current characteristics of the flow field inside the current interrogation window at location defined by the camera position (camera_position).

Example:

# Once the camera agent has been initialized:
ca = CameraAgentSingleDQN(...)

# And we have the set of cues computed, e.g., from the initial interrogation window:
camera_position, cues = ca.env.reset()

# We can use the epsilon-greedy strategy to choose the action:
ca.choose_action(cues=cues,
                 epsilon=0.5)
Parameters:

  • cues – numpy.ndarray specifying \(N\) cues that the RL agent senses. The cues are the input parameters to the Q-network. It has to have size (1, N).

  • epsilon – float specifying the exploration probability, \(\varepsilon\). It has to be between 0 and 1.

Returns:

  • action - int specifying the action selected.

pykitPIV.ml.CameraAgentSingleDQN.train(self, cues, action, reward, next_cues, new_learning_rate=0.001)

Trains the deep Q-network (q_network) with the outcome of a single step in the environment.

Note

This function uses tf.GradientTape() to record operations done on Q-network’s trainable parameters and to apply gradients.

Example:

# Once the camera agent has been initialized:
ca = CameraAgentSingleDQN(...)

# And we have access to cues, action, reward, and next_cues
# coming from the current step in the environment:
cues, action, reward, next_cues = ...

# We can train the agent with information about the current step in the environment:
ca.train(cues=cues,
         action=action,
         reward=reward,
         next_cues=next_cues,
         new_learning_rate=0.001)

To save the trained Q-network, you can run:

ca.q_network.save('SingleDQN.keras')
Parameters:

  • cues – numpy.ndarray specifying the current cues.

  • action – int specifying the current action selected.

  • reward – float specifying the current reward received.

  • next_cues – numpy.ndarray specifying the next cues seen after taking the current action in the environment.

  • new_learning_rate – (optional) float specifying the new learning rate to use in the current pass over the minibatch.

pykitPIV.ml.CameraAgentSingleDQN.view_weights(self)

Returns the latest weights and biases of the deep Q-network.

Example:

# Once the camera agent has been initialized:
ca = CameraAgentSingleDQN(...)

# View the current target Q-network weights:
ca.view_weights()
Returns:

  • weights_and_biases - list of numpy.ndarray specifying the current trainable parameters (weights and biases) of the deep Q-network.

Double deep Q-network training with memory replay

class pykitPIV.ml.CameraAgentDoubleDQN(env, target_q_network, online_q_network, memory_size=10000, batch_size=256, n_epochs=100, learning_rate=0.0001, optimizer='RMSprop', discount_factor=0.95, random_seed=None)

Creates a reinforcement learning (RL) agent that operates a virtual camera in a PIV experimental setting and provides a training loop for Q-learning. We use double Q-learning (see Hasselt et al. for more info) and use two deep neural network (DNN) models with memory replay. Having two Q-networks separates the process of finding which action has the maximum Q-value from the process of learning precisely what that maximum Q-value should be.

The goal of the RL agent is to learn the policy, which is mapping function, \(\pi\), from \(\text{cues} \rightarrow \text{actions}\) such that:

\[\text{action} = \pi(\text{cues} )\]

and where \(\pi\) is the trained DNN model.

Example:

from pykitPIV.ml import PIVEnv
from pykitPIV.ml import CameraAgent
import tensorflow as tf

# Initialize the environment:
env = PIVEnv(...)

# Create a simple neural network model for the Q-network:
class QNetwork(tf.keras.Model):

    def __init__(self, n_actions):

        super(QNetwork, self).__init__()

        self.dense1 = tf.keras.layers.Dense(10,
                                            activation='linear',
                                            kernel_initializer=tf.keras.initializers.Ones)
        self.dense2 = tf.keras.layers.Dense(10,
                                            activation='linear',
                                            kernel_initializer=tf.keras.initializers.Ones)
        self.output = tf.keras.layers.Dense(n_actions,
                                            activation='linear',
                                            kernel_initializer=tf.keras.initializers.Ones)

    def call(self, state):

        x = self.dense1(state)
        x = self.dense2(x)

        return self.output(x)

# Initialize the camera agent:
ca = CameraAgent(env=env,
                 target_q_network=QNetwork(env.n_actions),
                 online_q_network=QNetwork(env.n_actions),
                 memory_size=10000,
                 batch_size=256,
                 n_epochs=100,
                 learning_rate=0.0001,
                 optimizer='RMSprop',
                 discount_factor=0.95)
Parameters:

  • env – object of a custom environment class that is a subclass of gym.Env specifying the virtual environment. This can for instance be an object of the pykitPIV.ml.PIVEnv class.

  • target_q_network – tf.keras.Model specifying the deep neural network that will be the target (stable) network for Q-learning.

  • online_q_network – tf.keras.Model specifying the deep neural network that will be the online (temporary) network for Q-learning.

  • memory_size – (optional) int specifying the size of the memory bank.

  • batch_size – (optional) int specifying the batch size for training the Q-network for after each step in the environment.

  • n_epochs – (optional) int specifying the number of epochs to train the Q-network for after each step in the environment.

  • learning_rate – (optional) float specifying the initial learning rate. The learning rate can be updated on the fly by passing a new value to the train() function.

  • optimizer – (optional) str specifying the gradient descent optimizer to use. It can be 'Adam' or 'RMSprop'.

  • discount_factor – (optional) float specifying the discount factor, \(\gamma\).

  • random_seed – (optional) int specifying the random seed for random number generation in numpy. If specified, all operations are reproducible.

pykitPIV.ml.CameraAgentDoubleDQN.choose_action(self, cues, epsilon)

Defines an \(\varepsilon\)-greedy choice of the next best action to select.

If the probability is less than \(\varepsilon\), action is selected at random.

If the probability is higher than or equal to \(\varepsilon\), action is selected based on the cues that are the characteristic of the flow field inside the current interrogation window at location defined by the camera position (camera_position).

Example:

# Once the camera agent has been initialized:
ca = CameraAgent(...)

# And we have the set of cues computed, e.g., from the initial interrogation window:
camera_position, cues = ca.env.reset()

# We can use the epsilon-greedy strategy to choose the action:
ca.choose_action(cues=cues,
                 epsilon=0.5)
Parameters:

  • cues – numpy.ndarray specifying \(N\) cues that the RL agent senses. The cues are the input parameters to the Q-network. It has to have size (1, N).

  • epsilon – float specifying the exploration probability, \(\varepsilon\). It has to be between 0 and 1.

pykitPIV.ml.CameraAgentDoubleDQN.remember(self, cues, action, reward, next_cues)

Adds the complete outcome of taking a step in the environment to the memory bank. That outcome is represented by a tuple:

\[(\text{cues}, \text{action}, \text{reward}, \text{next cues})\]

where the cues change from \(\text{cues}\) to \(\text{next cues}\) after taking an action, and there is a numerical value of the reward associated with that action.

Example:

# Once the camera agent has been initialized:
ca = CameraAgent(...)

# We can reset the environment to initialize the interrogation window:
camera_position, cues = ca.env.reset()

# The typical interaction with the environment will be choosing an action and executing it,
# and remembering the outcome of that step by adding it to the memory bank.
# The code below can represent one episode:
for _ in range(0,100):

    action = ca.choose_action(cues,
                              epsilon=epsilon)

    next_camera_position, next_cues, reward = ca.env.step(action,
                                                          reward_function=reward_function,
                                                          reward_transformation=reward_transformation,
                                                          verbose=False)

    # We add the outcomes of the current step to the memory bank:
    ca.remember(cues,
                action,
                reward,
                next_cues)

    cues = next_cues
Parameters:

  • cues – numpy.ndarray specifying \(N\) cues that the RL agent senses before taking a step in the environment. The cues are the input parameters to the Q-network. It has to have size (1, N).

  • action – int specifying the action to be taken at the current step in the environment.

  • reward – float specifying the reward received after taking a step.

  • next_cues – numpy.ndarray specifying \(N\) cues that the RL agent senses after taking a step in the environment. The cues are the input parameters to the Q-network. It has to have size (1, N).

pykitPIV.ml.CameraAgentDoubleDQN.train(self, new_learning_rate=0.001)

Trains the temporary Q-network (online_q_network) with the outcome of a single step in the environment.

Example:

# Once the camera agent has been initialized:
ca = CameraAgent(...)

# We can train the agent with a single pass over a batch of training data:
ca.train(new_learning_rate=0.001)
Parameters:

new_learning_rate – (optional) float specifying the new learning rate to use in the current pass over the minibatch.

pykitPIV.ml.CameraAgentDoubleDQN.update_target_network(self)

Synchronizes the target Q-network with the online Q-network in double Q-learning (see Hasselt et al. for more information). This function can be called once every a couple of steps in the environment, or even once every a couple of episodes.

Example:

# The general abstraction for synchronizing the two Q-networks in a training loop
# can be the following:
for episode in range(0,n_episodes):

    ...

    # Synchronize the networks once every 10 episodes:
    if (episode+1) % 10 == 0:
        ca.update_target_network()
pykitPIV.ml.CameraAgentDoubleDQN.view_weights(self)

Returns the latest weights and biases of the target Q-network.

Example:

# Once the camera agent has been initialized:
ca = CameraAgent(...)

# View the current target Q-network weights:
ca.view_weights()

Reinforcement learning rewards

Note

The future version will also include unsupervised rewards computed directly on PIV image pairs, such as image frame consistency, smoothness, Charbonier loss, etc.

class pykitPIV.ml.Rewards(verbose=False, random_seed=None)

Provides custom reward functions that are applicable to various tasks related to navigating a virtual wind tunnel and a virtual PIV experiment.

Each function returns the reward, \(R\).

Functions of this class can be directly used as the reward_function parameter in pykitPIV.ml.PIVEnv.step().

Example:

from pykitPIV.ml import Rewards

# Instantiate an object of the Rewards class:
rewards = Rewards()
Parameters:

  • verbose – (optional) bool specifying if the verbose print statements should be displayed.

  • random_seed – (optional) int specifying the random seed for random number generation in numpy. If specified, all operations are reproducible.

pykitPIV.ml.Rewards.divergence(self, vector_field, transformation)

Computes the reward based on the divergence of the flow field.

Example:

from pykitPIV.ml import Rewards
import numpy as np

# Once we have the vector field specified
# e.g., velocity field or displacement field:
vector_field = ...

# Instantiate an object of the Rewards class:
rewards = Rewards(verbose=True,
                  random_seed=None)

# Design a custom transformation that looks for regions of high divergence (either positive or negative)
# and computes the maximum absolute value of divergence in that region:
def transformation(div):
    return np.max(np.abs(div))

# Compute the reward based on the divergence for the present velocity field:
reward = rewards.divergence(vector_field=vector_field,
                            transformation=transformation)
Parameters:

  • vector_field – numpy.ndarray specifying the vector field components under the interrogation window. It should be of size \((1, 2, H_{\text{i}}+2b, W_{\text{i}}+2b)\), where \(1\) is just one, fixed vector field, \(2\) refers to each vector field component. For example, it can be the velocity field with components \(u\) and \(v\), or the displacement field with components \(dx\) and \(dy\). \(H_{\text{i}}+2b\) is the height and \(W_{\text{i}}+2b\) is the width of the interrogation window.

  • transformation – function specifying an arbitrary transformation of the Q-criterion and an arbitrary compression of the Q-criterion field to a single value.

Returns:

  • reward - float specifying the reward, \(R\).

pykitPIV.ml.Rewards.vorticity(self, vector_field, transformation)

Computes the reward based on the vorticity of the flow field.

Example:

from pykitPIV.ml import Rewards
import numpy as np

# Once we have the vector field specified
# e.g., velocity field or displacement field:
vector_field = ...

# Instantiate an object of the Rewards class:
rewards = Rewards(verbose=True,
                  random_seed=None)

# Design a custom transformation that looks for regions of high vorticity (either positive or negative)
# and computes the mean absolute value of vorticity in that region:
def transformation(vort):
    return np.mean(np.abs(vort))

# Compute the reward based on the divergence for the present velocity field:
reward = rewards.vorticity(vector_field=vector_field,
                           transformation=transformation)
Parameters:

  • vector_field – numpy.ndarray specifying the vector field components under the interrogation window. It should be of size \((1, 2, H_{\text{i}}+2b, W_{\text{i}}+2b)\), where \(1\) is just one, fixed vector field, \(2\) refers to each vector field component. For example, it can be the velocity field with components \(u\) and \(v\), or the displacement field with components \(dx\) and \(dy\). \(H_{\text{i}}+2b\) is the height and \(W_{\text{i}}+2b\) is the width of the interrogation window.

  • transformation – function specifying an arbitrary transformation of the Q-criterion and an arbitrary compression of the Q-criterion field to a single value.

Returns:

  • reward - float specifying the reward, \(R\).

pykitPIV.ml.Rewards.q_criterion(self, vector_field, transformation)

Computes the reward based on the Q-criterion.

Example:

from pykitPIV.ml import Rewards
import numpy as np

# Once we have the vector field specified
# e.g., velocity field or displacement field:
vector_field = ...

# Instantiate an object of the Rewards class:
rewards = Rewards(verbose=True,
                  random_seed=None)

# Design a custom transformation that looks for vorticity-dominated regions
# (as opposed to shear-dominated regions)
# and computes the maximum value of the Q-criterion in that region:
def transformation(Q):
    Q = np.max(Q.clip(min=0))
    return Q

# Compute the reward based on the Q-criterion for the present velocity field:
reward = rewards.q_criterion(vector_field=vector_field,
                             transformation=transformation)
Parameters:

  • vector_field – numpy.ndarray specifying the vector field components under the interrogation window. It should be of size \((1, 2, H_{\text{i}}+2b, W_{\text{i}}+2b)\), where \(1\) is just one, fixed vector field, \(2\) refers to each vector field component. For example, it can be the velocity field with components \(u\) and \(v\), or the displacement field with components \(dx\) and \(dy\). \(H_{\text{i}}+2b\) is the height and \(W_{\text{i}}+2b\) is the width of the interrogation window.

  • transformation – function specifying an arbitrary transformation of the Q-criterion and an arbitrary compression of the Q-criterion field to a single value.

Returns:

  • reward - float specifying the reward, \(R\).

Reinforcement learning sensory cues

Note

The future version will also include unsupervised sensory cues computed directly on PIV image pairs, such as image frame consistency, smoothness, Charbonier loss, etc.

class pykitPIV.ml.Cues(verbose=False, random_seed=None, sample_every_n=10, normalize_displacement_vectors=False)

Provides custom sensory cues functions that are applicable to various tasks related to navigating a virtual wind tunnel and a virtual PIV experiment. The cue vector, \(\mathbf{c}\), is computed only based on the reconstructed displacement field, \(\vec{d\mathbf{s}}\), in the interrogation window.

Each function returns a vector of cues, \(\mathbf{c}\), that is a numpy.ndarray of \(N\) cues and of size \((1,N)\).

The sensory cues can become the inputs to Q-networks when training RL agents.

Functions of this class can be directly used as the cues_function parameter in pykitPIV.ml.PIVEnv.

Example:

from pykitPIV.ml import Cues

# Instantiate an object of the Cues class:
cues_obj = Cues(verbose=False,
                random_seed=None,
                sample_every_n=10,
                normalize_displacement_vectors=False)
Parameters:

  • verbose – (optional) bool specifying if the verbose print statements should be displayed.

  • random_seed – (optional) int specifying the random seed for random number generation in numpy. If specified, all operations are reproducible.

  • sample_every_n – (optional) int specifying the step in sampling the displacement vectors over a uniform grid.

  • normalize_displacement_vectors – (optional) bool specifying whether the (sampled) displacement vectors should be normalized to unit length.

pykitPIV.ml.Cues.sampled_vectors(self, displacement_field)

Computes the cues vector that contains \(n\) displacement vectors sampled on a uniform grid from the displacement field, \(\vec{d\mathbf{s}}\):

\[\vec{d\mathbf{s}}_1, \vec{d\mathbf{s}}_2, \dots, \vec{d\mathbf{s}}_n\]

These are further organized as:

\[\mathbf{c} = [dx_1, dx_2, \dots, dx_n, dy_1, dy_2, \dots, dy_n]\]

hence \(N = 2n\).

Example:

from pykitPIV.ml import Cues
import numpy as np

# Once we have the displacement field specified:
displacement_field = ...

# Instantiate an object of the Cues class:
cues_obj = Cues(verbose=True,
                random_seed=None,
                sample_every_n=10,
                normalize_displacement_vectors=False)

# Compute the cues vector:
cues = cues_obj.sampled_vectors(displacement_field=displacement_field)
Parameters:

displacement_field – numpy.ndarray specifying the velocity components under the interrogation window. It should be of size \((1, 2, H_{\text{i}}+2b, W_{\text{i}}+2b)\), where \(1\) is just one, fixed flow field, \(2\) refers to each velocity component \(u\) and \(v\) respectively, \(H_{\text{i}}+2b\) is the height and \(W_{\text{i}}+2b\) is the width of the interrogation window.

Returns:

  • cues - numpy.ndarray specifying the cues vector, \(\mathbf{c}\). It has shape \((1,N)\).

pykitPIV.ml.Cues.sampled_magnitude(self, displacement_field)

Computes the cues vector that contains \(N\) displacement magnitudes sampled on a uniform grid from the displacement field magnitude, \(|\vec{d\mathbf{s}}|\):

\[\mathbf{c} = [|\vec{d\mathbf{s}}_1|, |\vec{d\mathbf{s}}_2|, \dots, |\vec{d\mathbf{s}}_N|]\]

Example:

from pykitPIV.ml import Cues
import numpy as np

# Once we have the displacement field specified:
displacement_field = ...

# Instantiate an object of the Cues class:
cues_obj = Cues(verbose=True,
                random_seed=None,
                sample_every_n=10,
                normalize_displacement_vectors=False)

# Compute the cues vector:
cues = cues_obj.sampled_magnitude(displacement_field=displacement_field)
Parameters:

displacement_field – numpy.ndarray specifying the velocity components under the interrogation window. It should be of size \((1, 2, H_{\text{i}}+2b, W_{\text{i}}+2b)\), where \(1\) is just one, fixed flow field, \(2\) refers to each velocity component \(u\) and \(v\) respectively, \(H_{\text{i}}+2b\) is the height and \(W_{\text{i}}+2b\) is the width of the interrogation window.

Returns:

  • cues - numpy.ndarray specifying the cues vector, \(\mathbf{c}\). It has shape \((1,N)\).

pykitPIV.ml.Cues.sampled_divergence(self, displacement_field)

Computes the cues vector that contains \(N\) values for the divergence sampled on a uniform grid from the divergence of the displacement field, \(d = \nabla \cdot \vec{d\mathbf{s}}\):

\[\mathbf{c} = [d_1, d_2, \dots, d_N]\]

Example:

from pykitPIV.ml import Cues
import numpy as np

# Once we have the displacement field specified:
displacement_field = ...

# Instantiate an object of the Cues class:
cues_obj = Cues(verbose=True,
                random_seed=None,
                sample_every_n=10,
                normalize_displacement_vectors=False)

# Compute the cues vector:
cues = cues_obj.sampled_divergence(displacement_field=displacement_field)
Parameters:

displacement_field – numpy.ndarray specifying the velocity components under the interrogation window. It should be of size \((1, 2, H_{\text{i}}+2b, W_{\text{i}}+2b)\), where \(1\) is just one, fixed flow field, \(2\) refers to each velocity component \(u\) and \(v\) respectively, \(H_{\text{i}}+2b\) is the height and \(W_{\text{i}}+2b\) is the width of the interrogation window.

Returns:

  • cues - numpy.ndarray specifying the cues vector, \(\mathbf{c}\). It has shape \((1,N)\).

pykitPIV.ml.Cues.sampled_vorticity(self, displacement_field)

Computes the cues vector that contains \(N\) values for the vorticity sampled on a uniform grid from the vorticity of the displacement field, \(\omega = \nabla \times \vec{d\mathbf{s}}\):

\[\mathbf{c} = [\omega_1, \omega_2, \dots, \omega_N]\]

Example:

from pykitPIV.ml import Cues
import numpy as np

# Once we have the displacement field specified:
displacement_field = ...

# Instantiate an object of the Cues class:
cues_obj = Cues(verbose=True,
                random_seed=None,
                sample_every_n=10,
                normalize_displacement_vectors=False)

# Compute the cues vector:
cues = cues_obj.sampled_vorticity(displacement_field=displacement_field)
Parameters:

displacement_field – numpy.ndarray specifying the velocity components under the interrogation window. It should be of size \((1, 2, H_{\text{i}}+2b, W_{\text{i}}+2b)\), where \(1\) is just one, fixed flow field, \(2\) refers to each velocity component \(u\) and \(v\) respectively, \(H_{\text{i}}+2b\) is the height and \(W_{\text{i}}+2b\) is the width of the interrogation window.

Returns:

  • cues - numpy.ndarray specifying the cues vector, \(\mathbf{c}\). It has shape \((1,N)\).

pykitPIV.ml.Cues.sampled_q_criterion(self, displacement_field)

Computes the cues vector that contains \(N\) values for the Q-criterion sampled on a uniform grid from the Q-criterion of the displacement field, \(Q\):

\[\mathbf{c} = [Q_1, Q_2, \dots, Q_N]\]

Example:

from pykitPIV.ml import Cues
import numpy as np

# Once we have the displacement field specified:
displacement_field = ...

# Instantiate an object of the Cues class:
cues_obj = Cues(verbose=True,
                random_seed=None,
                sample_every_n=10,
                normalize_displacement_vectors=False)

# Compute the cues vector:
cues = cues_obj.sampled_q_criterion(displacement_field=displacement_field)
Parameters:

displacement_field – numpy.ndarray specifying the velocity components under the interrogation window. It should be of size \((1, 2, H_{\text{i}}+2b, W_{\text{i}}+2b)\), where \(1\) is just one, fixed flow field, \(2\) refers to each velocity component \(u\) and \(v\) respectively, \(H_{\text{i}}+2b\) is the height and \(W_{\text{i}}+2b\) is the width of the interrogation window.

Returns:

  • cues - numpy.ndarray specifying the cues vector, \(\mathbf{c}\). It has shape \((1,N)\).

Reinforcement learning utilities

pykitPIV.ml.plot_trajectory(trajectory, quantity=None, vector_field=None, interrogation_window_size=None, interrogation_window_size_buffer=None, c_path='white', c_init='white', c_final='black', s=10, lw=2, xlabel=None, ylabel=None, xticks=True, yticks=True, cmap='viridis', vmin=None, vmax=None, add_quiver=False, quiver_step=10, quiver_color='k', quiver_linewidths=1, add_streamplot=False, streamplot_density=1, streamplot_color='k', streamplot_linewidth=1, figsize=(10, 5), dpi=300, filename=None)

Plots the trajectory taken by the trained RL agent in a new environment.

Example:

from pykitPIV import plot_trajectory

# Assuming that we will plot the displacement field magnitude as the scalar quantity:
displacement_field_magnitude = ...

# And we will plot the displacement field as the vector field:
displacement_field = ...

# And we have computed the trajectory matrix:
trajectory = ...

# We can visualize the trajectory in the virtual environment:
plot_trajectory(trajectory,
                quantity=displacement_field_magnitude,
                vector_field=displacement_field,
                interrogation_window_size=(60,60),
                interrogation_window_size_buffer=10,
                c_path='white',
                c_init='white',
                c_final='black',
                s=10,
                lw=2,
                xlabel=None,
                ylabel=None,
                xticks=True,
                yticks=True,
                cmap='viridis',
                vmin=None,
                vmax=None,
                add_quiver=False,
                quiver_step=10,
                quiver_color='k',
                quiver_linewidths=1,
                add_streamplot=False,
                streamplot_density=1,
                streamplot_color='k',
                streamplot_linewidth=1,
                figsize=(10, 5),
                dpi=300,
                filename=None)

One example of plotted trajectory after training the agent is:

../_images/ml_plot_trajectory.png
Parameters:

  • trajectory – numpy.ndarray specifying the camera trajectory taken by the agent in the 2D virtual wind tunnel. It should be of size \((n_s, 2)\), where \(n_s\) is the number of steps taken in the environment, and columns represent the camera position along the height and width of the virtual wind tunnel, respectively.

  • quantity – (optional) numpy.ndarray specifying the scalar quantity to visualize in the background. It should be of size \((H_{\text{wt}}, W_{\text{wt}})\).

  • vector_field – (optional) numpy.ndarray specifying the vector field to visualize on top of any scalar quantity. It should be of size \((1, 2, H_{\text{wt}}, W_{\text{wt}})\).

  • interrogation_window_size – (optional) tuple of two int elements specifying the size of the interrogation window in pixels \([\text{px}]\). The first number is the window height, \(H_{\text{i}}\), the second number is the window width, \(W_{\text{i}}\).

  • interrogation_window_size_buffer – (optional) int specifying the buffer, \(b\), in pixels \([\text{px}]\) to add to the interrogation window size in the width and height direction.

  • c – (optional) str specifying the color for the interrogation window outline.

  • s – (optional) int or float specifying the size of the dot that represents the camera position.

  • lw – (optional) int or float specifying the line width for the interrogation window outline.

  • xlabel – (optional) str specifying \(x\)-label.

  • ylabel – (optional) str specifying \(y\)-label.

  • xticks – (optional) bool specifying if ticks along the \(x\)-axis should be plotted.

  • yticks – (optional) bool specifying if ticks along the \(y\)-axis should be plotted.

  • cmap –

    (optional) str or an object of matplotlib.colors.ListedColormap specifying the color map to use.

  • vmin – (optional) int or float specifying the minimum on the colorbar.

  • vmax – (optional) int or float specifying the maximum on the colorbar.

  • normalize_cbars – (optional) bool specifying if the colorbar for the interrogation window should be normalized to the colorbar for the entire wind tunnel.

  • add_quiver – (optional) bool specifying if vector field should be plotted on top of the scalar magnitude field.

  • quiver_step – (optional) int specifying the step on the pixel grid to attach a vector to. The higher this number is, the less dense the vector field is.

  • quiver_color – (optional) str specifying the color of vectors.

  • quiver_linewidths – (optional) int or float specifying the line widths of vectors.

  • add_streamplot – (optional) bool specifying if streamlines should be plotted on top of the scalar magnitude field.

  • streamplot_density – (optional) float or int specifying the streamplot density.

  • streamplot_color – (optional) str specifying the streamlines color.

  • streamplot_linewidth – (optional) int or float specifying the line width for the streamplot.

  • figsize – (optional) tuple of two numerical elements specifying the figure size as per matplotlib.pyplot.

  • dpi – (optional) int specifying the dpi for the image.

  • filename – (optional) str specifying the path and filename to save an image. If set to None, the image will not be saved.

Returns:

  • plt - matplotlib.pyplot image handle.

  • cbar - matplotlib.pyplot colorbar handle.