pynif3d.aggregation

class pynif3d.aggregation.NeRFAggregator(background_color=None, noise_std=0.0)

Bases: torch.nn.modules.module.Module

Color aggregation function. Takes the raw predictions obtained from a NIF model, the depth values and a ray direction to produce the final color of a pixel. Please refer to the original NeRF paper for more information.

Usage:

# Assume the models are given.
nif_model = NeRFModel()
renderer = PointRenderer()
aggregator = NeRFAggregator()

# Get the RGBA values for the corresponding points and view directions.
rgba_values = renderer(nif_model, query_points, view_directions)

# Aggregate the computed RGBA values to form the RGB maps, depth maps etc.
rendered_data = aggregator(rgba_values, ray_z_values, ray_directions)
rgb_map, depth_map, disparity_map, alpha_map, weight = rendered_data
Parameters

  • background_color (torch.Tensor) – The background color to be added for rendering. If set to None, no background will be added. Default is None.

  • noise_std (float) – The standard deviation of the noise to be added to alpha. Set 0 to disable the noise addition. Default is 0.

forward(raw_model_prediction, z_vals, rays_d)
Parameters

  • raw_model_prediction (torch.Tensor) – The prediction output of a NIF model. Its shape is (number_of_rays, number_of_points_per_ray, 4).

  • z_vals (torch.Tensor) – Depth values of the rays. Its shape is (number_of_rays, number_of_points_per_ray, 4).

  • rays_d (torch.Tensor) – The direction vector of the rays. Its shape is (number_of_rays, 3).

Returns

Tuple containing:

  • rgb_map (torch.Tensor): The RGB pixel values of the rays. Its shape is (number_of_rays, 3).

  • depth_map (torch.Tensor): The depth pixel values of the rays. Its shape is (number_of_rays,).

  • disparity_map (torch.Tensor): The inverse depth pixel values of the rays. Its shape is (number_of_rays,).

  • alpha_map (torch.Tensor): The transparency/alpha value of each pixel. Its shape is (number_of_rays,).

  • weights (torch.Tensor): The weights of each sampled points across a ray which impact the final RGB/depth/disparity/transparency value of a pixel. Its shape is (number_of_rays, number_of_points_per_ray).

Return type

tuple

training: bool