Skip to content

Commit

Permalink
Merge pull request #4 from KonradDanielewski/refactor
Browse files Browse the repository at this point in the history 
refactor
  • Loading branch information
@KonradDanielewski
KonradDanielewski authored Feb 9, 2024
2 parents 78041b0 + 285511a commit 9dbd451
Show file tree
Hide file tree
Showing 22 changed files with 1,980 additions and 1,424 deletions.
22 changes: 22 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
seba/__pycache__/__init__.cpython-310.pyc
seba/__pycache__/version.cpython-310.pyc
seba/data/__pycache__/__init__.cpython-310.pyc
seba/data/__pycache__/analysis.cpython-310.pyc
seba/data/__pycache__/auxfun_data.cpython-310.pyc
seba/data/__pycache__/behavior_io.cpython-310.pyc
seba/data/__pycache__/ephys.cpython-310.pyc
seba/data/__pycache__/histology.cpython-310.pyc
seba/data/__pycache__/io.cpython-310.pyc
seba/plotting/__pycache__/__init__.cpython-310.pyc
seba/plotting/__pycache__/auxfun_ploting.cpython-310.pyc
seba/plotting/__pycache__/plotting_funcs.cpython-310.pyc
seba/utils/__pycache__/__init__.cpython-310.pyc
seba/utils/__pycache__/auxfun_data.cpython-310.pyc
seba/utils/__pycache__/auxfun_ephys.cpython-310.pyc
seba/utils/__pycache__/auxfun_plotting.cpython-310.pyc
seba/utils/__pycache__/auxiliary.cpython-310.pyc
SEBA.egg-info/dependency_links.txt
SEBA.egg-info/PKG-INFO
SEBA.egg-info/requires.txt
SEBA.egg-info/SOURCES.txt
SEBA.egg-info/top_level.txt
269 changes: 221 additions & 48 deletions example_script/Example_notebook.ipynb
View file
Original file line number Diff line number Diff line change
@@ -1,123 +1,296 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "15f91571",
"metadata": {},
"source": [
"## SEBA example workflow \n",
"\n",
"This notebook covers the usage of most high level functions in SEBA. It consists of:\n",
"1. Reading and formatting behavioral annotation data from Boris/BehaView into text files with onset/offset timestamps synchronized to ephys time\n",
"\n",
"2. Preparing a binary data structure (event is represented according to the recording where onset and offset is either read from data - for behaviors or provided otherwise by the user) that can easily be used for further modelling approaches like [CEBRA](https://github.com/AdaptiveMotorControlLab/CEBRA)\n",
" \n",
"3. Analyzing the data by calculating firing rate, z-score for each event\n",
"\n",
"4. Applying Wilcoxon rank sum test to establish neuron responsivness to each event\n",
"\n",
"5. Using histology data from [HERBS](https://github.com/JingyiGF/HERBS) to match each neuron to a brain region it was recorded from\n",
"\n",
"6. Create plots to explore the data:\n",
" - Matrix of common neurons between the events\n",
" - Matrix of N neurons responding to event vs structure\n",
" - Heatmaps of neurons per recording or for all recordings for comparison of overall population activity to a specific event\n",
" - PSTH plots to explore neural activity for each event\n",
" - Paired heatmaps for comparison of overall population activity to two different events per recording or for all recordings\n",
" - Paired PSTH plots to compare activity of specific neurons to 2 different events"
]
},
{
"cell_type": "markdown",
"id": "003d9469",
"metadata": {},
"source": [
"### Import necessary packages"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"id": "38766583",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from glob import glob\n",
"import seba"
"import seba\n",
"import pandas as pd"
]
},
{
"cell_type": "markdown",
"id": "e40b16ab",
"metadata": {},
"source": [
"### Set paths to where the data is and where it should be saved"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 2,
"id": "2bf4b525",
"metadata": {},
"outputs": [],
"source": [
"data_folder = r\"C:\\Ephys_analysis\\data\\Ephys\" #folder that contains folders with ephys recordings\n",
"histology_folder = r\"C:\\Ephys_analysis\\data\\NP_tiled\" #folder that contains folders with HERBS output after drawing probes onto the images\n",
"filepaths = glob(r\"C:\\Ephys_analysis\\data\\behav_annot_old\\*.bvs\") #path to behaview output of behaview behavior annotation\n",
"framert = [30, 50, 50, 50, 50, 50] #list of framerates of videos\n",
"save_path = glob(r\"C:\\Ephys_analysis\\data\\Ephys\\*\") #list of folders with ephys_recordings\n",
"save_results=r\"C:\\Ephys_analysis\\data\\Results_test\" #folder in which the data structure and plots will be saved"
"data_folder = glob(r\"path_to_dir\\*\") # folder that contains folders with ephys recordings\n",
"histology_data = r\"path_to_dir\" # folder which contains all folders (per recording) generated by HERBS\n",
"save_results = r\"path_to_dir\" # folder in which the data structure and plots will be saved if dir doesn't exist it will be created"
]
},
{
"cell_type": "markdown",
"id": "0bd1959b",
"metadata": {},
"source": [
"### Set behavior annotation data paths"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "35ad8369",
"id": "b65b631d",
"metadata": {},
"outputs": [],
"source": [
"#If using BehaView\n",
"for file, fps, save in zip(filepaths, framert, save_path): \n",
" seba.data.io.read_bvs(filepath=file, framerate=fps, save_path=save)\n",
"seba.extract_raw_events_TS_BehaView(data_folder=data_folder, framerate=30, append_CS=True, CS_filename=\"TTL_3.txt\", CS_len=20)"
"# NOTE: The way to get the path usually needs to be more specific especially when it comes to the file names for BORIS since csv is a common format\n",
"bvs = glob(r\"path_to_dir\\*.bvs\")\n",
"boris = glob(r\"path_to_dir\\*.csv\")"
]
},
{
"cell_type": "markdown",
"id": "3eedd936",
"metadata": {},
"source": [
"### Read and transform behavior annotations, append any external events"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a7a8bd82",
"id": "6b1a623d",
"metadata": {},
"outputs": [],
"source": [
"#If using boris annotations - assumes that for each recording annotations from boris were manually moved to corresponding folder\n",
"seba.read_extract_boris(data_folder=data_folder, boris_output=\"boris_output.csv\", \n",
" camera_TTL=\"cam_TTL.txt\", append_CS=True, CS_filename=\"TTL_3.txt\", CS_len=20, padding=False) "
"# Example usage. NOTE: the file have to be matched to their recording directiories. It's easiest if the files are actually stored with the ephys recording itself\n",
"for file, save in zip(bvs, data_folder[:3]):\n",
" seba.read_bvs(save, file, onsets_only=True)\n",
"\n",
"for file, save in zip(boris, data_folder[3:]):\n",
" seba.read_boris(save, file, onsets_only=True)\n",
"\n",
"# NOTE: CS.txt specifies the name of the file that contains an external event. Name is assumed to be the same for each recording as it should represent the same type of an event\n",
"for dir in data_folder:\n",
" seba.append_event_to_binary(dir, os.path.join(dir, r\"CS.txt\"), 20) "
]
},
{
"cell_type": "markdown",
"id": "4d1d01e6",
"metadata": {},
"source": [
"### If not all recordings contain all events, specify the event names. \n",
"\n",
"<b>IMPORTANT:</b> Otherwise it's assumed that the first recording contains all possible events and names are derived from it!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fbbfbe97",
"id": "38c589be",
"metadata": {},
"outputs": [],
"source": [
"#Get histology data\n",
"#If using neuropixels 2.0. If not, skip first function\n",
"seba.fix_wrong_shank_NP2(data_folder)\n",
"seba.get_brain_regions(data_folder, histology_folder, neuropixels_20=True)"
"event_names = [\n",
" \"cs_onsets\", \n",
" \"freezing_experimental_onsets\",\n",
" \"freezing_partner_onsets\",\n",
" \"rearing_experimental_onsets\",\n",
" \"rearing_partner_onsets\",\n",
" \"relaxed_experimental_onsets\",\n",
" \"relaxed_partner_onsets\",\n",
" \"social_both_onsets\",\n",
"]"
]
},
{
"cell_type": "markdown",
"id": "f379ae17",
"metadata": {},
"source": [
"### Read ephys data, perform analysis and create a data structure to store it. \n",
"\n",
"To check all `seba.structurize_data` attributes call `seba.structurize_data?`\n",
"\n",
"If `calculate_responsive=True` it will perform Wilcoxon rank sum test and append information about neuron responsivness to the data structure."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "38c589be",
"id": "7bea658a",
"metadata": {},
"outputs": [],
"source": [
"#Build data structure\n",
"data_obj = seba.structurize_data(data_folder, save_path=save_results, pre_event=2, post_event=4, bin_size=0.01, calculate_responsive=True, p_bound=0.05)\n",
"#Save information about responsivity to cluster_info_good.csv\n",
"# Build and save data structure\n",
"data_obj = seba.structurize_data(\n",
" data_folder=data_folder,\n",
" event_names=event_names,\n",
" save_path=save_results,\n",
" pre_event=2,\n",
" post_event=4,\n",
" bin_size=0.02,\n",
" calculate_responsive=True,\n",
" p_bound=0.05,\n",
" spike_threshold=5,\n",
")\n",
"\n",
"# Save information about responsivity to cluster_info_good.csv\n",
"seba.responsive_neurons2events(data_folder, data_obj)"
]
},
{
"cell_type": "markdown",
"id": "60fb15c9",
"metadata": {},
"source": [
"### Read HERBS data and append information about brain regions to cluster_info_good.csv"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dff4e471",
"metadata": {},
"outputs": [],
"source": [
"seba.get_brain_regions(data_folder, histology_folder, neuropixels_20=True | False)\n",
"\n",
"seba.fix_wrong_shank_NP2(data_folder) # Take only the folders that use NP 2.0. Should be checked if new versions of HERBS are being used."
]
},
{
"cell_type": "markdown",
"id": "75722f09",
"metadata": {},
"source": [
"### Add brain region information to data_obj"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cf2c37ec",
"metadata": {},
"outputs": [],
"source": [
"seba.add_brain_regions(data_folder, data_obj)"
]
},
{
"cell_type": "markdown",
"id": "b7a9ea77",
"metadata": {},
"source": [
"### If running the notebook again and ephys_data.pickle already exist read it with the cell below"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "cfbaeb83",
"metadata": {},
"outputs": [],
"source": [
"#data_obj = pd.read_pickle(r\"path_to_ephys_data.pickle\")"
]
},
{
"cell_type": "markdown",
"id": "548e7428",
"metadata": {},
"source": [
"### Define pairs of behaviors you want to explore. Those will be used for paired plots"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f862ef87",
"metadata": {},
"outputs": [],
"source": [
"#Define pairs of behaviors you want to explore\n",
"behavioral_pairs = [[\"freezing_ephys_onsets\", \"freezing_non_ephys_onsets\"],\n",
" [\"rearing_ephys_onsets\", \"rearing_non_ephys_onsets\"],\n",
" [\"freezing_ephys_onsets\", \"CS_onsets\"],\n",
" [\"rearing_ephys_onsets\", \"social_onsets\"],\n",
" [\"freezing_ephys_offsets\", \"freezing_non_ephys_offsets\"],\n",
" [\"rearing_ephys_offsets\", \"rearing_non_ephys_offsets\"],\n",
" [\"freezing_ephys_offsets\", \"CS_offsets\"],\n",
" [\"rearing_ephys_offsets\", \"social_offsets\"],\n",
" [\"freezing_ephys_onsets\", \"freezing_ephys_offsets\"],\n",
" [\"rearing_ephys_onsets\", \"rearing_ephys_offsets\"],\n",
" [\"CS_onsets\", \"CS_offsets\"],\n",
" [\"social_onsets\", \"social_offsets\"],\n",
"\n",
"behavioral_pairs = [[\"freezing_experimental_onsets\", \"freezing_partner_onsets\"],\n",
" [\"freezing_experimental_onsets\", \"relaxed_experimental_onsets\"],\n",
" [\"rearing_experimental_onsets\", \"rearing_partner_onsets\"],\n",
" ]"
]
},
{
"cell_type": "markdown",
"id": "8634e784",
"metadata": {},
"source": [
"### Run plotting\n",
"\n",
"`per_recording`, `z_score`, `responsive_only` are main attributes users can modify where:\n",
"1. `per_recording=True` means that data is plotted per recording, otherwise it's plotted for all recordings combined\n",
"2. `z_score=True` means that z-scored data is plotted, otherwise firing rate is used\n",
"3. `responsive_only=True` means that only responsive neurons are taken into account when creating plots. Otherwise activity of all neurons is plotted.\n",
"4. `y_limit` specified the y axis limits for line plots (PSTH)\n",
"5. `percent_all_neurons=True` means that the matrix will show a percentage of all neurons for recording/all recordings (depends on `per_recording`) instead of the number of neuorns"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fbd3c688",
"metadata": {},
"outputs": [],
"source": [
"seba.plot_common_nrns_matrix(data_obj, save_results, per_animal=False, percent_all_neurons=False)\n",
"#Plots everything\n",
"seba.plot_common_nrns_matrix(data_obj, save_results, per_recording=False, percent_all_neurons=True)\n",
"seba.plot_common_nrns_matrix(data_obj, save_results, per_recording=True, percent_all_neurons=True)\n",
"for pair in behavioral_pairs:\n",
" seba.plot_heatmaps_paired(data_obj, pair, save_path=save_results, per_animal=False, responsive_only=True, colormap = \"inferno\", z_score=True)\n",
" seba.plot_lin_reg_scatter(data_obj, pair, save_path=save_results, per_animal=False, responsive_only=True)\n",
" seba.plot_psths_paired(data_obj, pair, save_path=save_results, responsive_only=True, z_score=True, ylimit= [-1, 3])\n",
" seba.plot_heatmaps_paired(data_obj, pair, save_path=save_results, per_recording=False, responsive_only=True, colormap = \"inferno\", z_score=True)\n",
" seba.plot_lin_reg_scatter(data_obj, pair, save_path=save_results, per_recording=False, responsive_only=True)\n",
" seba.plot_psths_paired(data_obj, pair, save_path=save_results, responsive_only=True, z_score=True, ylimit= [-2, 4])\n",
"\n",
"seba.plot_psths(data_obj, save_results, ylimit= [-1, 3], responsive_only=True)\n",
"seba.plot_psths(data_obj, save_results, ylimit= [-2, 4], responsive_only=True)\n",
"seba.plot_heatmaps(data_obj, save_results, responsive_only=True)\n",
"seba.neurons_per_structure(data_folder, data_obj, save_results)\n",
"seba.neurons_per_event(data_folder, data_obj, save_results)"
"seba.plot_neurons_per_event_structure(data_folder, data_obj, save_results, per_recording=False)"
]
}
],
Expand All @@ -137,7 +310,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.0"
"version": "3.10.13"
}
},
"nbformat": 4,
Expand Down
Loading

0 comments on commit 9dbd451

Please sign in to comment.