Automated analysis of courtship suppression learning and memory in Drosophila melanogaster

Abstract

Study of the fruit fly, Drosophila melanogaster, has yielded important insights into the underlying molecular mechanisms of learning and memory. Courtship conditioning is a well-established behavioral assay used to study Drosophila learning and memory. Here, we describe the development of software to analyze courtship suppression assay data that correctly identifies normal or abnormal learning and memory traits of individual flies. Development of this automated analysis software will significantly enhance our ability to use this assay in large-scale genetic screens and disease modeling. The software increases the consistency, objectivity, and types of data generated.

Keywords: :

• courtship suppression
• learning
• memory
• automated behavior analysis
• feature detection
• clustering
• video processing

Introduction

Considerable insight into the underlying molecular mechanisms of learning and memory has been made by studying the fruit fly, Drosophila melanogaster.^1-5 This is largely due to the variety of molecular tools and behavioral assays that are available to study fly learning and memory (reviewed in ³). In addition to the extensive genetic tools available in the fly to analyze the neural circuitry associated with learning and memory,^6-9 both adult and larval flies exhibit a number of behaviors that can be altered with training.³ One of these well-established behaviors is courtship conditioning.

Courting behavior by males in Drosophila follows a linear, stereotyped, and well documented set of behaviors,¹⁰ and these behaviors are modified by previous sexual experience.^11,12 Courtship conditioning is a form of associative learning in Drosophila, where male courtship behavior is modified by exposure to a previously mated female that is unreceptive to courting.^11,13 Thus, after 1 h of courting a mated female, males suppress their courtship behavior even toward subsequent receptive virgin females for 1–3 h.¹¹ This courtship suppression is measured by the Courtship Index (CI), which is calculated by dividing the total amount of time each male fly spends courting by the total duration of a testing period.^11,13 A CI decrease in response to such training is “indicative of behavioral plasticity in the form of memory”¹⁴.

Courtship conditioning is a powerful tool for learning and memory research. A practical limitation of this assay is the time required to visually inspect the courtship behavior when evaluating CI. In order to measure learning during training and memory requires observing one fly for three ten-minute periods, spread over at least one hour and up to several weeks, depending on the type of memory being assessed. These practical considerations make using courtship conditioning during large-scale screening for genes and/or molecules difficult. Further, manual analysis may be insensitive to behaviors that are too rapid, too subtle or too complex to measure. Development of reliable computer software that is capable of analyzing courtship conditioning would significantly enhance our ability to utilize this behavior when testing learning and memory in large-scale genetic screens and disease modeling, and would increase both the consistency and the objectivity of the data generated.

Automated courtship behavior analysis has been previously described for Drosophila.^15,16 However, the utilization of this technology for assessing courtship suppression in learning during training and memory, specifically how this analysis compares to manual courtship suppression analysis for leaning and memory, is unknown. To address these problems, we created a novel software system to analyze courtship behavior for use in learning and memory studies. Automated analysis of videos that captures courting flies produces motion feature vectors that characterize and quantify the behavior of the flies. From these feature vectors a Computed Courtship Index (CCI), a computational equivalent of the existing Courtship Index, may be calculated. Clustering techniques, e.g., k-means clustering,¹⁷ may also be applied to the feature vectors in order to computationally group fly specimens by phenotype, e.g., learning during training and memory capabilities. We report here that our software is capable of discriminating between normal learning during training and memory of this training, and abnormal learning during training and memory using the courtship conditioning assay for testing.

Results

To determine if the software was able to automatically generate a CCI comparable to a traditional CI value for flies capable of both learning and memory, we analyzed 210 total videos from 70 individual flies that showed normal and abnormal learning during training and memory. We calculated the CI and CCI for each of these flies, and subjected each to arcsin square root transformations to approximate normal distributions (see Materials and Methods). We selected 36 different flies from two genotypes (see Materials and Methods) that individually showed normal learning during training and memory by manual CI analysis (Fig. 2A and B, white columns). Manual analysis of these flies showed a significant decrease in their overall courtship behavior in the last ten minutes of a one-hour training session, compared with the first 10 min (Fig. 2A, white columns, p = 2.98 × 10⁻¹⁷). Further, trained flies showed a significant reduction in courtship behavior as compared with sham-trained flies, indicative of normal immediate-recall memory of training (Fig. 2B, white columns, p = 2.07 × 10⁻⁹).

Figure 2. Software is capable of measuring successful learning with successful memory in Drosophila courtship suppression. (A) denotes learning ability during the first 10 min and last 10 min of the courtship suppression assay training phase. Grey bars indicate courtship index (CI) as computed by the software program and subsequently subjected to arcsin square root transformation. White bars indicate CI as computed manually as previously described,²⁴ and subsequently subjected to arcsin square root transformation. Note significant learning is measured by both methods. n = 36. (B) denotes memory ability of trained flies compared with sham trained flies. Grey bars indicate CI as computed by the software program and subsequently subjected to arcsin square root transformation. White bars indicate CI as computed manually, as previously described,²⁴ and subsequently subjected to arcsin square root transformation. Note successful memory in both cases. n = 30. Error bars represent ± SEM. *p < 0.05.

Similar to the manual CI analysis, our software also calculated a significant decrease in CCI in the last ten minutes of a one-hour training session, compared with the first 10 min (Fig. 2A, gray columns, p = 3.80 × 10⁻¹⁰). Further, trained flies analyzed by our software also showed a significant reduction in courtship behavior as compared with sham-trained flies (Fig. 2B, gray columns, p = 0.0001). There is no significant difference between the CIs calculated manually from those calculated by the software in this analysis. This is true for the CIs calculated in the first 10 min (p = 0.127), the last 10 min (p = 0.110), the trained flies (p = 0.057), and the sham flies (p = 0.053). Taken together, these data suggest that our software is able to identify both successful learning, as well as memory from those flies capable of performing this behavior robustly.

We next determined whether the software could also detect flies with memory deficits. We selected 25 different flies from the two genotypes that individually showed a successful learning during training responses, but no memory of that training. In both cases, the manual and computed CIs showed a significant difference between the first and last 10 min of training (Fig. 3A, manual p = 1.58 × 10⁻⁶, computed p = 0.0001), indicative of a successful learning during training. However, for both the manual and computed CIs, there was no significant change in CIs from trained animals compared with sham controls (Fig. 3B, manual p = 0.153, computed p = 0.113), indicating that these animals have no memory of training. Again, there is no significant difference between the CIs calculated manually from those calculated by the software in this analysis. This is true for the CIs calculated in the first 10 min (p = 0.977), the last 10 min (p = 0.412), the trained flies (p = 0.984), and the sham flies (p = 0.606). Taken together, these data suggest that our software program is also capable of identifying unsuccessful memory in the context of successful learning.

Figure 3. Software is capable of measuring successful learning with unsuccessful memory in Drosophila courtship suppression. Grey bars indicate courtship index (CI) as computed by the software program and subsequently subjected to arcsin square root transformation in each panel. White bars indicate CI as computed manually in each panel, as previously described,²⁴ and subsequently subjected to arcsin square root transformation. (A) denotes learning ability during the first 10 min and last 10 min of the courtship suppression assay training phase. Note significant learning is measured by both methods. n = 25. (B) denotes memory ability of trained flies compared with sham trained flies. Note unsuccessful memory in both cases. n = 25. Error bars represent ± SEM. *p < 0.05.

In the previous analyses, we selected individual flies that exhibit a specific, desired behavioral response. From these flies the derived data show that our software is capable of producing a CI value similar to that of a human observer. We next wanted to determine if the computer software could identify differences among groups itself, with no prior human intervention. To accomplish this, we turned to cluster analysis and increased the number of individual flies used in the experiment. We took 59 flies from the various genotypes with a range of CIs for both learning during training and memory.. We subjected their memory feature vectors to k-means cluster analysis,¹⁷ which separated these vectors into two groups based on the memory capabilities of the flies. In order to gauge how well our software was able to separate out flies with different memory capabilities, we calculated both the CI and the CCI for each of the computed groups (Fig. 4). Based on the resulting values, it was clear that the two groups represented flies with good memory (Fig. 4A, CCI p = 0.0013), and poor memory (Fig. 4B, CCI p = 0.321). There were no significant differences between the CI and the CCI between flies displaying either good memory (p = 0.564) or poor memory (p = 0.372). We next performed this same cluster analysis based on the learning feature vectors of the flies, and calculated both the CI and the CCI for each of these different computed groups. Again, it was clear that these two groups represented flies with good learning (Fig. 4C, CCI p = 9.68 × 10⁻⁹), and poor learning (Fig. 4D, CCI p = 0.320). This analysis does not take into account the genotype of the fly, only its learning and memory capabilities. Again, there was no significant difference between the CI and CCI between either good learners (p = 0.504) and bad learners (p = 0.733). Taken together, our data suggests that our software is capable of successfully clustering two groups of flies based solely on their behavior with no prior human intervention.

Figure 4. Software is capable of detecting both successful and unsuccessful learning and memory without prior human intervention. Flies are computationally clustered into two groups using geometric feature vectors automatically derived from videos. The two groups in each case are as follows: (A) Flies with successful memory; (B) flies with no memory; (C) flies with successful learning; and (D) flies with no learning. Grey bars indicate CI as computed by the software program and subsequently subjected to arcsin square root transformation for each group. White bars indicate CI as computed manually for each group, and subsequently subjected to arcsin square root transformation. (A) denotes memory ability of trained flies compared with sham trained flies. This group exhibits successful memory in both cases. n = 29. (B) denotes memory ability of trained flies compared with sham trained flies. This group exhibits unsuccessful memory in both cases. n = 28. (C) denotes learning ability during the first 10 min and last 10 min of the courtship suppression assay training phase. This group exhibits significant learning in both cases. n = 31. (D) denotes learning ability during the first 10 min and last 10 min of the courtship suppression assay training phase. This group exhibits unsuccessful learning in both cases. n = 26. Error bars represent ± SEM. *p < 0.05.

Discussion

The development of tracking software for Drosophila melanogaster is not novel,^15,18 and publicly available software programs, such as Ctrax, exist to facilitate this analysis (http://ctrax.berlios.de). However, due to the range and complexity of movements associated with fly courtship, along with the number of specific actions that must be captured to utilize this behavior for learning and memory assessment, the question as to whether the analysis of this behavior can indeed be automated remained outstanding. Here, we present evidence that software designed to analyze this behavior is capable of successfully detecting clear differences in learning and memory capability among genetically diverse flies. Our data suggest that this software is sensitive enough to identify differences in behavior, so much so that it can distinguish between those flies that are both capable and incapable of successful learning and memory. However, we would like to point out that our software is not capable of traditional binning of individual fly behaviors in order to track the percentage of time a male fly spends at each stage of the courtship ritual. This would require more sophisticated methods of behavior detection, which our software lacks. However, we do anticipate that this software analysis may be capable of distinguishing between more subtle differences in behavior among groups of individual flies. Analyzing larger samples of flies and clustering them into several groups may allow for a more detailed analysis of learning and memory in both wild type and mutant flies. Finally, our software can also allow for increased throughput of mutant or pharmacological screening in Drosophila models of human disease, such as Alzheimer disease.

Materials and Methods

Drosophila stocks and genetics

Unless otherwise noted, all crosses were performed at 25°C on standard cornmeal-molasses-agar medium. BL# refers to Bloomington Stock Center stock number. Stocks used are P{GawB} elav¹⁵⁵ (Elav-Gal4, BL# 458); P{UAS:APP} and P{UAS:BACE}, and are described in reference 19.

We raised flies of two different genotypes that included: 1) wild type (Canton S), and 2) flies expressing the human Alzheimer disease proteins APP and BACE (Elav-Gal4; UAS:APP; UAS:BACE, as described in ref. 20). Individual flies from each of these genotypes showed both normal and abnormal learning and memory. However, flies expressing UAS:APP and UAS:BACE showed abnormal memory as a group.²⁰

We utilized these two different genotypes to assure that we were able to obtain sufficient numbers of individual flies that both showed normal learning and memory (wild type), and abnormal learning and memory (Alzheimer disease model flies). Because we were testing the methodology of utilizing a computer program for learning and memory assays, and not testing for a learning and memory difference in these genotypes per se, we mixed the videos from each of these genotypes based on successful learning and memory responses from individual flies.

Behavioral testing and training

For all behavioral tests, flies were maintained at 25°C in a 12:12 light:dark cycle at 60% humidity. Courtship analysis scored by human eye was performed as previously described in reference 20. Briefly, virgin male flies were collected between 0 and 6 h after eclosion and transferred to individual food vials. Behavior was digitally captured using a Sony DCR-SR47 Handycam with Carl Zeiss optics. Subsequent digital video analysis of time spent performing courtship behavior was quantified using iMovie software (Apple). The total time that a male performed courtship activity was measured and scored. The Courtship Index (CI) was calculated as the total time observed performing courting behavior divided by the total time assayed, as described in reference 11.

Virgin female wild type (Canton S) flies were collected and kept in normal food vials in groups of 10. Male flies were aged for 3 d before behavioral training and testing. All tests were performed during the relative light phase. Mated Canton S females used for training were 5 d old, and observed to have mated with a Canton S male the evening prior to training. Virgin female Canton S targets used were 4 d old. Male flies were assigned to random groups the day of training, and assays were set up and scored blind. Male flies were transferred without anesthesia to one half of a partitioned mating chamber from Aktogen (www.aktogen.com) that contained a previously mated Canton S female in the other partitioned half. Males were allowed to acclimate for 1 min, then the partition between the male and female was removed. Male flies were then trained for 60 min. After 60 min, male flies were transferred within 2 min without anesthesia to one half of a clean partitioned mating chamber that contained a virgin Canton S female in the other partitioned half. The partition was removed and the flies were recorded for 10 min.

All CI and CCI values were subjected to arcsin square root transformations to approximate normal distributions, as described in reference 14. To determine significance among the same individuals for the learning phase of this assay, a one-tailed paired Student’s t-test was performed on the transformed data. To determine significance among different individuals of the same gentotype a one-tailed unpaired Student’s t-test was performed on transformed data. To determine the difference between computer generated and manual data, a one way ANOVA was performed on transformed data. Tukey post-hoc tests were used, unless the data did not meet the homogeneity of variances assumption, in which case the Games Howell post-hoc test was used.

Video analysis, quantification and classification

Our software system is capable of tracking and measuring the motions of Drosophila melanogaster recorded in videos under fixed laboratory conditions. We used the quantitative data derived from the videos to calculate a CCI (Computed CI) and to group the flies based on eight behavioral characteristics. The software system consists of several computational stages that include extraction/cropping of the video image frames, segmentation, filtering, tracking, geometric computation, and finally CCI calculation or clustering (grouping). The extraction/cropping stage takes a video sequence as input (10 min in duration for our initial study) that may contain several experimental chambers. The stage produces a series of images (1 per frame), which are then cropped based on user input to isolate the individual chambers in separate images (Fig. 1A). The segmentation stage first computes a background image from each sequence of images associated with a chamber, i.e., an experimental sample. Given the background image, the pixels associated with the flies in the chamber may by identified and grouped, producing a binary image where groups of white pixels are labeled as foreground (fly) and black pixels are labeled as background. The filtering stage removes noise (groups of white pixels smaller than a threshold value) from the binary images, and fills voids within the groups. The two largest groups of white pixels are identified in each image and represent the two flies at that moment in time (Fig. 1B). We are also able to determine when two flies are touching each other. This produces a single large pixel group, rather than two smaller groups. In our initial efforts an attempt was made to distinguished between male and female flies based on their size, but we found in our experiments that physical size alone could not consistently be used to identify the gender of a fly.

Figure 1. Schematic of video capture and assessment. (A) Representative image from a courtship video used for analysis in this study. Two individual frames are shown. (B) Representative binary image produced by background subtraction and filtering. (C) Representative image for male/female identification and tracking.

Once two groups of pixels are calculated for each image in a video sequence, they may be tracked from frame to frame. The pixel groups are identified and labeled in each subsequence of an analyzed video. A subsequence is a contiguous series of images where each image contains two fly objects (white pixel groups). A subsequence with fewer than 15 images (i.e., half of the frame-rate) is not processed. The Euclidean distance between the centroids of the two fly objects in each subsequence image is computed. The frame with the maximum inter-centroid distance is identified as a starting image for this subsequence. From the starting image, a frame-to-frame correspondence for each white pixel group is attributed to the remaining images in the subsequence. In other words, each white region in the starting image is associated with a fly and this association is propagated in the forward and backward directions in the range of the subsequence. A minimum distance criterion is used for establishing the fly object correspondence.

A four-way distance calculation between the two fly objects in the currently processed image and the two fly objects in the adjacent image is performed. The minimum of the four distances determines the correspondence between a fly object in one image with a fly object in an adjacent image. The labels for the first and second white regions of the starting image are initialized to be A and B respectively. Each pixel group in the adjacent image is given the label of the closest group in the starting image. Using this minimum distance criterion the labels are propagated to the remaining images in the subsequence.

The output of the tracking stage produces a series of binary images that contains either one or two groups of white pixels that have been labeled Fly A and Fly B, along with several geometric quantities computed for each pixel group in each image. Figure 1C demonstrates that the output may be used to visually label the flies. The green line displays the centroid and head direction of the flies. The male (smaller) fly is marked with a yellow circle. The tracking output provides the input for the geometric computation stage. This stage calculates a set of geometric quantities each for Fly A and Fly B from the whole sequence of images. The quantities include: (1) percentage of frames when one fly is looking at another, (2) percentage of frames when flies are together, (3) distribution of the distances between the fly centroids, (4) distribution of the head direction angles between flies, and (5) distribution of the flies’ speeds. From these five quantities an 8-dimensional feature vector can be calculated for each processed video sequence. The eight features included in the vector are: (1) percentage of frames when one fly is looking at the other, (2) percentage of frames when flies are together, (3) mean of inter-centroid distances, (4) standard deviation of inter-centroid distances, (5) mean of head direction angles, (6) standard deviation of head direction angles, (7) mean of the flies’ speeds multiplied by 100, and (8) standard deviation of the flies’ speeds multiplied by 100. The last two numbers in the feature vector are multiplied by 100 so that all eight numbers have the same order of magnitude. More information about the video/image analysis algorithms and the calculation of features can be found in references 21 and 22. Interested parties may request the software by contacting the corresponding author.

The features based on inter-centroid distance, head direction angles and fly velocities were straightforward to compute, and therefore included in our feature vector. The movement and activity of the flies are related to the courtship behavior itself. These features capture general information about the flies’ movements. The percentage of time together and the percentage time when flies are looking at each other are known to contribute to the manual CI determination, so those were also included in our feature vector. We did not test our approach with other combinations of features because the eight features we used allowed us to successfully classify flies based on behavior and produce an acceptable computational equivalent of the standard manual CI. Further, many of these features are related to courtship. For example, one of the most significant behaviors that courting male flies perform is following the female as she moves away from them. This behavior is captured by the inter-centroid distance and head direction angle features. Similarly, a male fly orientating, or “looking at” the female is also significant. The computer takes these features into consideration when computing a CCI value. Future work will explore other features, as well as include an analysis of the contribution and significance of each feature to the CCI and clustering calculations.

Once a feature vector is calculated for each video sequence, a least squares optimization process may be used to fit a straight line to the vectors in order to produce a CCI. The CCI is a number, based solely on the computerized analysis of the videos, that quantifies the flies’ courtship behavior. The CCI is a linear combination of the computed feature values, where the weights (coefficients) associated with each feature are calculated, using singular value decomposition (SVD), to minimize || Ax – b ||₂, where ||•||₂ is the Euclidean norm, A is an N × 8 dimensional matrix containing N feature vectors, x is the coefficient vector, and b is a column vector whose elements are the manually-derived CI values for each of the samples. Once x has been computed from a set of samples the CCI for one of the samples can be computed from the formula:

CCI = PLA × x₁ + PFT × x₂ + MCD × x₃ + SDCD × x₄ + MHA × x₅ + SDHA × x₆ + MS × x₇ + SDS × x₈,

where PLA is the value of the first feature percentage of frames when one fly is looking at the other, PFT is the second feature, and so on, and the x_i’s are the computed coefficients.

As mentioned above, this software calculates 5 geometric quantities that are used to create 8-dimensional feature vectors for each video sequence. Independent of CCI, the raw feature vector data may also be utilized to perform cluster analysis and classification. By applying k-means clustering²³ to the feature vectors it is possible to identify and group the samples in terms of learning and memory behaviors based only on the geometric features extracted from the videos. When clustering samples by learning ability the feature vectors calculated from a sample’s last 10 min of a one-hour training session is subtracted from the feature vector computed from the first 10 min. This is done because the difference between a fly’s first and last 10 min behaviors characterizes its ability to learn. This produces a Learning feature vector for each fly. Similarly, when clustering samples by memory ability the feature vector for a trained fly is subtracted from the average vector calculated from all of the feature vectors for sham trained flies. This produces a Memory feature vector for each fly. During cluster analysis the Euclidean norm is used to calculate the distance between two feature vectors.

Our linear combination does not guarantee a CCI value between 0 and 100. A few of our CCI calculations (less than 5%) produce values less than 0 and greater than 100. These values are clamped to lie within the range 0 to 100. Due to the arcsin transformations performed during statistical analysis, these values become greater than 100. However, from a biological point of view, it is more important that the software generate data that shows a trend that allows the user to determine learning and memory behavior rather than to produce CCIs within a normal range of 0–100.

Acknowledgments

We would like to thank the members of the Marenda and Saunders labs for their helpful contributions to this paper. This work was supported by grants from the NIH: R01NS057295 (A.J.S.) and R21RR026074 (D.R.M.).

Disclosure of Potential Conflicts of Interest

No potential conflicts of interest were disclosed.