[2024-06-25 18:15:35,332] : DEBUG : viirs_edr_geotiff : _prepare_initial_logging : Starting script with arguments: -r viirs_edr -w geotiff -p true_color_surf -f ../edr/SurfRefl_v1r2_j01_s202406051850316_e202406051851561_c202406052248020.nc ../edr/SurfRefl_v1r2_j01_s202406051851574_e202406051853201_c202406052248060.nc ../edr/SurfRefl_v1r2_j01_s202406051853213_e202406051854458_c202406052248120.nc ../edr/SurfRefl_v1r2_j01_s202406051854471_e202406051856116_c202406052248150.nc ../edr/SurfRefl_v1r2_j01_s202406051856128_e202406051857374_c202406052248130.nc ../edr/SurfRefl_v1r2_j01_s202406051857386_e202406051859031_c202406052248110.nc ../edr/SurfRefl_v1r2_j01_s202406051859044_e202406051900271_c202406052248070.nc -vv [2024-06-25 18:15:35,332] : INFO : viirs_edr_geotiff : _run_processing : Sorting and reading input files... [2024-06-25 18:15:35,332] : DEBUG : viirs_edr_geotiff : _set_preferred_chunk_size : Setting preferred chunk size to 6400 pixels or 312MiB [2024-06-25 18:15:35,342] : DEBUG : satpy.readers.yaml_reader : load_yaml_configs : Reading ('/data/users/kathys/polar2grid_v_3_1/libexec/python_runtime/lib/python3.11/site-packages/satpy/etc/readers/viirs_edr.yaml',) [2024-06-25 18:15:35,348] : DEBUG : satpy.readers.yaml_reader : create_filehandlers : Assigning to viirs_edr: ['../edr/SurfRefl_v1r2_j01_s202406051859044_e202406051900271_c202406052248070.nc', '../edr/SurfRefl_v1r2_j01_s202406051857386_e202406051859031_c202406052248110.nc', '../edr/SurfRefl_v1r2_j01_s202406051853213_e202406051854458_c202406052248120.nc', '../edr/SurfRefl_v1r2_j01_s202406051856128_e202406051857374_c202406052248130.nc', '../edr/SurfRefl_v1r2_j01_s202406051854471_e202406051856116_c202406052248150.nc', '../edr/SurfRefl_v1r2_j01_s202406051851574_e202406051853201_c202406052248060.nc', '../edr/SurfRefl_v1r2_j01_s202406051850316_e202406051851561_c202406052248020.nc'] [2024-06-25 18:15:35,713] : PID 2195585 : DEBUG : polar2grid.core.script_utils : rename_log_file : Log renamed from '/data/users/kathys/test_data/viirs_edr/j01/new/viirs_edr_geotiff_fail.log' to 'viirs_edr_geotiff_20240605_185031.log' [2024-06-25 18:15:35,713] : PID 2195585 : INFO : viirs_edr_geotiff : _run_processing : Loading product metadata from files... [2024-06-25 18:15:35,713] : PID 2195585 : DEBUG : satpy.composites.config_loader : load_compositor_configs_for_sensor : Looking for composites config file viirs.yaml [2024-06-25 18:15:35,865] : PID 2195585 : DEBUG : pyorbital.tlefile : _get_config_path : Path to the Pyorbital configuration (where e.g. platforms.txt is found): /data/users/kathys/polar2grid_v_3_1/libexec/python_runtime/lib/python3.11/site-packages/pyorbital/etc [2024-06-25 18:15:35,906] : PID 2195585 : DEBUG : satpy.composites.config_loader : load_compositor_configs_for_sensor : Looking for composites config file visir.yaml [2024-06-25 18:15:35,996] : PID 2195585 : DEBUG : satpy.readers.yaml_reader : _load_dataset_area : No coordinates found for DataID(name='longitude_750', resolution=750, modifiers=()) [2024-06-25 18:15:36,027] : PID 2195585 : DEBUG : satpy.readers.yaml_reader : _load_dataset_area : No coordinates found for DataID(name='latitude_750', resolution=750, modifiers=()) [2024-06-25 18:15:36,059] : PID 2195585 : DEBUG : satpy.readers.yaml_reader : _load_dataset_area : No coordinates found for DataID(name='latitude_375', resolution=375, modifiers=()) [2024-06-25 18:15:36,090] : PID 2195585 : DEBUG : satpy.readers.yaml_reader : _load_dataset_area : No coordinates found for DataID(name='longitude_375', resolution=375, modifiers=()) [2024-06-25 18:15:36,238] : PID 2195585 : INFO : viirs_edr_geotiff : _persist_swath_definition_in_scene : Loading swath geolocation into memory... [2024-06-25 18:15:47,586] : PID 2195585 : DEBUG : viirs_edr_geotiff : _persist_swath_definition_in_scene : 2 unique swath definitions persisted [2024-06-25 18:15:47,588] : PID 2195585 : DEBUG : satpy.scene : _generate_composite : Delaying generation of DataID(name='true_color_surf') because of incompatible areas [2024-06-25 18:15:47,588] : PID 2195585 : WARNING : satpy.scene : _remove_failed_datasets : The following datasets were not created and may require resampling to be generated: DataID(name='true_color_surf') [2024-06-25 18:15:47,588] : PID 2195585 : INFO : polar2grid.filters._filter_scene : _filter_scene_day_only_products : Running day coverage filtering... [2024-06-25 18:15:47,589] : PID 2195585 : DEBUG : polar2grid.filters._base : filter_scene : Analyzing 'DataID(name='surf_refl_M03', wavelength=WavelengthRange(min=0.478, central=0.488, max=0.498, unit='µm'), resolution=750, modifiers=())' for filtering... [2024-06-25 18:15:47,634] : PID 2195585 : WARNING : polar2grid.filters._utils : boundary_for_area : Geolocation data contains invalid bounding values. Computing entire array to get bounds instead. [2024-06-25 18:15:47,725] : PID 2195585 : DEBUG : polar2grid.filters.day_night : _filter_data_array : Sunlight is estimated to cover 100.00% of the scene. [2024-06-25 18:15:47,728] : PID 2195585 : DEBUG : polar2grid.filters._base : filter_scene : Analyzing 'DataID(name='surf_refl_M04', wavelength=WavelengthRange(min=0.545, central=0.555, max=0.565, unit='µm'), resolution=750, modifiers=())' for filtering... [2024-06-25 18:15:47,728] : PID 2195585 : DEBUG : polar2grid.filters._base : filter_scene : Analyzing 'DataID(name='surf_refl_M05', wavelength=WavelengthRange(min=0.662, central=0.672, max=0.682, unit='µm'), resolution=750, modifiers=())' for filtering... [2024-06-25 18:15:47,728] : PID 2195585 : DEBUG : polar2grid.filters._base : filter_scene : Analyzing 'DataID(name='surf_refl_I01', wavelength=WavelengthRange(min=0.6, central=0.64, max=0.68, unit='µm'), resolution=375, modifiers=())' for filtering... [2024-06-25 18:15:47,774] : PID 2195585 : WARNING : polar2grid.filters._utils : boundary_for_area : Geolocation data contains invalid bounding values. Computing entire array to get bounds instead. [2024-06-25 18:15:47,893] : PID 2195585 : DEBUG : polar2grid.filters.day_night : _filter_data_array : Sunlight is estimated to cover 100.00% of the scene. [2024-06-25 18:15:47,894] : PID 2195585 : INFO : polar2grid.filters._filter_scene : _filter_scene_night_only_products : Running night coverage filtering... [2024-06-25 18:15:47,895] : PID 2195585 : DEBUG : polar2grid.filters._base : filter_scene : Analyzing 'DataID(name='surf_refl_M03', wavelength=WavelengthRange(min=0.478, central=0.488, max=0.498, unit='µm'), resolution=750, modifiers=())' for filtering... [2024-06-25 18:15:47,895] : PID 2195585 : DEBUG : polar2grid.filters._base : filter_scene : Analyzing 'DataID(name='surf_refl_M04', wavelength=WavelengthRange(min=0.545, central=0.555, max=0.565, unit='µm'), resolution=750, modifiers=())' for filtering... [2024-06-25 18:15:47,895] : PID 2195585 : DEBUG : polar2grid.filters._base : filter_scene : Analyzing 'DataID(name='surf_refl_M05', wavelength=WavelengthRange(min=0.662, central=0.672, max=0.682, unit='µm'), resolution=750, modifiers=())' for filtering... [2024-06-25 18:15:47,895] : PID 2195585 : DEBUG : polar2grid.filters._base : filter_scene : Analyzing 'DataID(name='surf_refl_I01', wavelength=WavelengthRange(min=0.6, central=0.64, max=0.68, unit='µm'), resolution=375, modifiers=())' for filtering... [2024-06-25 18:15:48,226] : PID 2195585 : DEBUG : root : _areas_to_resample : Using default resampling target area 'wgs84_fit'. [2024-06-25 18:15:48,227] : PID 2195585 : DEBUG : polar2grid.resample._resample_scene : resample_scene : Products to preserve resolution for: set() [2024-06-25 18:15:48,227] : PID 2195585 : DEBUG : polar2grid.resample._resample_scene : resample_scene : Products to use new resolution for: {DataID(name='true_color_surf')} [2024-06-25 18:15:48,227] : PID 2195585 : INFO : polar2grid.resample._resample_scene : _freeze_area_if_dynamic : Computing dynamic grid parameters... [2024-06-25 18:15:49,472] : PID 2195585 : DEBUG : polar2grid.resample._resample_scene : _freeze_area_if_dynamic : Frozen dynamic area: Area ID: wgs84_fit Description: Longitude/Latitude WGS84 Grid Projection: {'datum': 'WGS84', 'no_defs': 'None', 'proj': 'longlat', 'type': 'crs'} Number of columns: 7281 Number of rows: 6776 Area extent: (-110.808, 8.7609, -69.3063, 47.3841) [2024-06-25 18:15:49,474] : PID 2195585 : INFO : polar2grid.resample._resample_scene : _resample_scene_to_single_area : Resampling to 'wgs84_fit' using 'ewa' resampling... [2024-06-25 18:15:49,474] : PID 2195585 : DEBUG : polar2grid.resample._resample_scene : _resample_scene_to_single_area : Resampling to 'wgs84_fit' using resampler 'ewa' with {'cache_dir': None, 'persist': False, 'weight_delta_max': 40.0, 'weight_distance_max': 2.0} [2024-06-25 18:15:49,475] : PID 2195585 : DEBUG : satpy.scene : _resampled_scene : Resampling DataID(name='surf_refl_I01', wavelength=WavelengthRange(min=0.6, central=0.64, max=0.68, unit='µm'), resolution=375, modifiers=()) [2024-06-25 18:15:49,475] : PID 2195585 : INFO : satpy.scene : _reduce_data : Not reducing data before resampling. [2024-06-25 18:15:49,487] : PID 2195585 : DEBUG : satpy.scene : _resampled_scene : Resampling DataID(name='surf_refl_M03', wavelength=WavelengthRange(min=0.478, central=0.488, max=0.498, unit='µm'), resolution=750, modifiers=()) [2024-06-25 18:15:49,487] : PID 2195585 : INFO : satpy.scene : _reduce_data : Not reducing data before resampling. [2024-06-25 18:15:49,497] : PID 2195585 : DEBUG : satpy.scene : _resampled_scene : Resampling DataID(name='surf_refl_M04', wavelength=WavelengthRange(min=0.545, central=0.555, max=0.565, unit='µm'), resolution=750, modifiers=()) [2024-06-25 18:15:49,497] : PID 2195585 : INFO : satpy.scene : _reduce_data : Not reducing data before resampling. [2024-06-25 18:15:49,504] : PID 2195585 : DEBUG : satpy.scene : _resampled_scene : Resampling DataID(name='surf_refl_M05', wavelength=WavelengthRange(min=0.662, central=0.672, max=0.682, unit='µm'), resolution=750, modifiers=()) [2024-06-25 18:15:49,504] : PID 2195585 : INFO : satpy.scene : _reduce_data : Not reducing data before resampling. [2024-06-25 18:15:49,521] : PID 2195585 : DEBUG : satpy.composites : _get_and_sharpen_rgb_data_arrays_and_meta : Sharpening image with high resolution red band [2024-06-25 18:15:49,563] : PID 2195585 : DEBUG : satpy.scene : unload : Unloading dataset: DataID(name='surf_refl_I01', wavelength=WavelengthRange(min=0.6, central=0.64, max=0.68, unit='µm'), resolution=375, modifiers=()) [2024-06-25 18:15:49,563] : PID 2195585 : DEBUG : satpy.scene : unload : Unloading dataset: DataID(name='surf_refl_M03', wavelength=WavelengthRange(min=0.478, central=0.488, max=0.498, unit='µm'), resolution=750, modifiers=()) [2024-06-25 18:15:49,564] : PID 2195585 : DEBUG : satpy.scene : unload : Unloading dataset: DataID(name='surf_refl_M04', wavelength=WavelengthRange(min=0.545, central=0.555, max=0.565, unit='µm'), resolution=750, modifiers=()) [2024-06-25 18:15:49,564] : PID 2195585 : DEBUG : satpy.scene : unload : Unloading dataset: DataID(name='surf_refl_M05', wavelength=WavelengthRange(min=0.662, central=0.672, max=0.682, unit='µm'), resolution=750, modifiers=()) [2024-06-25 18:15:49,564] : PID 2195585 : DEBUG : polar2grid.utils.legacy_compat : apply_p2g_name_to_scene : Mapping Satpy ID to P2G name: DataID(name='true_color_surf', resolution=375) -> true_color_surf [2024-06-25 18:15:49,566] : PID 2195585 : DEBUG : satpy.writers : read_writer_config : Reading ['/data/users/kathys/polar2grid_v_3_1/libexec/python_runtime/lib/python3.11/site-packages/satpy/etc/writers/geotiff.yaml'] [2024-06-25 18:15:50,109] : PID 2195585 : DEBUG : satpy.writers : add_config_to_tree : Adding enhancement configuration from file: /data/users/kathys/polar2grid_v_3_1/libexec/python_runtime/lib/python3.11/site-packages/satpy/etc/enhancements/generic.yaml [2024-06-25 18:15:50,223] : PID 2195585 : DEBUG : satpy.writers : add_config_to_tree : Adding enhancement configuration from file: /data/users/kathys/polar2grid_v_3_1/etc/polar2grid/enhancements/generic.yaml [2024-06-25 18:15:50,238] : PID 2195585 : DEBUG : satpy.writers : add_config_to_tree : Adding enhancement configuration from file: /data/users/kathys/polar2grid_v_3_1/libexec/python_runtime/lib/python3.11/site-packages/satpy/etc/enhancements/viirs.yaml [2024-06-25 18:15:50,263] : PID 2195585 : DEBUG : satpy.writers : add_config_to_tree : Adding enhancement configuration from file: /data/users/kathys/polar2grid_v_3_1/etc/polar2grid/enhancements/viirs.yaml [2024-06-25 18:15:50,263] : PID 2195585 : DEBUG : satpy.writers : apply : Data for DataID(name='true_color_surf', resolution=375) will be enhanced with options: [{'name': 'reflectance_range', 'method': , 'kwargs': {'stretch': 'crude', 'min_stretch': -1.0, 'max_stretch': 110.0}}, {'name': 'Linear interpolation', 'method': , 'kwargs': {'xp': [0.0, 25.0, 55.0, 100.0, 255.0], 'fp': [0.0, 90.0, 140.0, 175.0, 255.0], 'reference_scale_factor': 255}}] [2024-06-25 18:15:50,263] : PID 2195585 : DEBUG : trollimage.xrimage : stretch : Applying stretch crude with parameters {'min_stretch': -1.0, 'max_stretch': 110.0} [2024-06-25 18:15:50,267] : PID 2195585 : DEBUG : satpy.enhancements : piecewise_linear_stretch : Applying the piecewise_linear_stretch [2024-06-25 18:15:50,416] : PID 2195585 : INFO : viirs_edr_geotiff : _run_processing : Computing products and saving data to writers... [2024-06-25 18:16:06,613] : PID 2195585 : INFO : viirs_edr_geotiff : _run_processing : SUCCESS