We, the NeuroData Family, each agree to all of the below commitments. These agreements are designed to facilitate succeeding at our mission while in alignment with our values. They are in a continual process of refinement; in life, work, and science, we experiment, learn, and grow. Note that these agreements augment, and therefore do not replace, official JHU policies, including: 1. The rights and responsibilities of all JHU students (documented here). 2. JHU advisor-PhD student relationships commitments set forth by the University Provost. 3. University's policies on Academic Freedom.
Joining the lab, which is made official by signing documents with JHU where JHU commits to you, amounts to also agreeing to all of the below core agreements, and Jovo commits to the supervisor commitments vis-a-vis you. In other words, failure to comply with the core agreements is sufficient grounds to find other opportunities for you that do not involve being a member of the lab. There is no judgement associated with not adhering to these agreements, though they are for everyone in the lab, they are not for everyone in the world.
We agree to focus the majority of our energy to research and to be present during typical working hours. Specifically, this means we will not schedule regularly recurring non-research activities (such as social engagements, sports, or other hobbies) during typical working hours, except for dealing with your dependents, PI/co-advisor organized events, or other explicit exceptions. We will also not invest more energy in extra-curricular activities than we do on research.
And, if the lab's projects are not sufficient for you, in one way or another, it is totally ok to work on non-lab stuff. If you find yourself in that situation, consider talking to labmates and/or jovo to make sure you are getting satisfaction.
Maximally Intrinsically Motivated for Successful Mission Critical Research
We agree to focus our research on projects that we are maximally intrinsically motivated to complete, are likely to be successful given our resources, and are mission critical.
In the context of each of our research goals, we may have many different interests. The mission of NeuroData, as stated in our mission statement, is to understand and improve machine and animal intelligences, worldwide. The research projects that we primarily work on, while on the NeuroData team, satisfy the following three criteria:
- the project is highly significant with respect to our mission,
- given my current resources, my estimate of the probability that I can satisfactorily complete this project is high (which includes shooting for the stars and landing on the moon, or even a gently rolling hill; but excludes shooting for the stars and crashing into the ocean), and
- in the set of every considered research project that I believe is feasible, this project is near the top of that list sorted by how much intrinsic motivation I feel with respect to each.
If I were more motivated to work on some other problem outside the purview of the mission of NeuroData, could not find a project that was highly significant to the NeuroData mission, or I could not find a project that I was likely to succeed it, then I will find another primary research team to work with (though the NeuroData team would love to remain friends). See how to choose a project for more tips on how to choose a project.
We agree to focus on 2 projects, unless otherwise negotiated. In general, each project should take about 1 year to complete and is complete when the manuscript is published in a journal/conference. One project is the main focus that we are actively working, the other is more exploratory, that will lead to a similarly large project in the following year. Individual variation is possible with suitable justification.
Open & Reproducible Science
We will conduct our work using the highest open and reproducible science standards. This means that our work will result in open source code, open access data derivatives, and open access publication (posting to a pre-print server) no later than journal submission, and ideally, throughout the entire scientific process. No later than the week of submitting a manuscript, it will be posted to pre-print server. Code will only be not open source if there are external reasons (e.g., patient privacy, external collaborator insistence). This also includes adopting best practices for diversity and inclusion in all of our activities. For example, see [https://github.com/mozilla/inclusion] for some examples with respect to code, and we make figures that are color-blind sensitive, use >10 point font, etc. All repo's that we maintain officially (i.e., that are mentioned in a pre-print/paper) will be maintained according to FIRM standards.
Crucially, posting to pre-print servers, and submitting to journals, do not explicitly require supervisor approval of the content. Rather, assuming the recommended guidelines are followed, then all lab co-authors have 1 week to provide feedback and modification requests. After another week passes, the manuscript may be submitted and the pre-print may be posted, even in the absence of further feedback from any lab members (including jovo). Note that collaborators outside the lab are not beholden to this agreement. If for some reason a lab member cannot be reached during this time, we can still post/submit, but agree to remove said lab person from the co-author list if they so choose within 1 week of their return. One week extensions can be granted to lab members if necessary, but only one week extension is permissable.
Prompt feedback is critical both for producing a quality manuscript and for sustaining forward movement on a project. Our lab-internal one-week feedback policy is intended to keep projects moving forward AND solicit the necessary feedback to produce high-quality papers. Importantly, this is only an expectation NeuroData members have agreed to meet. When NeuroData members are the first and/or last authors of a manuscript, consider requesting your collaborators also adhere to this lab policy. When a NeuroData member is a middle author on an outside group's manuscript, proactively request to review any drafts prior to submission and promise feedback in no more than one week. If a shorter timeline is required, try to be as accommodating as possible. As individuals and as a lab, it is our professional responsibility to know what is in a manuscript that carries our names, so it is appropriate to request that co-authors share final manuscripts with us prior to submission.
When it is time to start writing a paper, clone our latex template overleaf repo, invite jovo to it and make him owner so that compilation goes faster and we can track changes. If a collaborator demands google docs for some reason, and we are not senior authors, then we will have to acquiese to their wishes. In general, when anybody gives feedback on the paper, make sure track changes is on. I recommend giving feedback via the following convention:
% TODO blah blah ---jovo
This enables everyone to easily find all the comments, including if they are editing locally (or simply not in the overleaf web app). When feedback is provided, it is important to address every single comment in the text itself. Recall: the author of a manuscript is literally the least qualified person in the world to ascertain whether what that person has written is clear to other people, which is the question at hand. Therefore, if you are the author, and you think it was clear, you are wrong, in that it certainly was not clear to at least one person that cares.
Everyone is welcome to use either AWS or Azure as desired. Currently, we have free Azure credits, so that is the preference. For AWS, if you want to use it, the prerequisite is writing a AWS Cloud Credits for Research grant. Request just under $10,000 for small projects, and request up to $100,000 for larger ones. For the Brainlit project, we have an NIH grant, and that is what we should be using entirely until it runs out of money. However, anyone who is using cloud computing should not run any code that runs overnight without verifying the code is correct. It must be run on small parameters/dataset, and a figure must be generated on that small run which needs to be approved by the PI.
We agree that when we reach a certain level of seniority on the team (have successfully submitted >=1 first author peer-reviewed manuscript on our team), we will be open to accepting a "buddy" mentee. Our responsibility associated with this mentee will be to onboard the mentee. There is lots of literature to suggest this is good for everyone on the team, for example, see here and here.
We agree to reach out to potential collaborators when we find any information that may be useful for the lab such as algorithms or public code we want to incorporate into our package. Invite them to co-author any papers we produce, and in general include them in our community.
We endeavor to provide a safe and welcoming community for BIPOC, URM, LGBTQ+, students with disabilities, and any individual who self-identifies as part of a historically and contemporaneously marginalized community. We recognize the unique struggles of being a member of a marginalized community (and all the more so for individuals that are members of multiple such communities) in STEM and are committed to creating a environment where everyone feels supported, appreciated, and welcome. This includes:
- accomodating student needs so that they feel safe in lab and on campus
- connecting student with relevant affinity groups on campus
To support this goal we encourage all lab members to attend diversity training opportunities hosted by the university (e.g., Safe Zone Training). Our purpose in attending diversity training is to gain knowledge and skills we can use to provide a maximally welcoming and productive lab environment for people of all backgrounds. We hope to convey our commitment to this type of inclusive environment by outwardly displaying lab members' completion of such training on the lab website.
We agree to provide a harrassment-free experience for everyone (see wikipedia's definition of harrassment for details), regardless of the following: age, disability, physical appearance, body size, race, ethnicity, nationality, sex, sexual orientation, gender identity and expression, religion, technical/biological experience.
Sexual language and imagery is not appropriate. Racial microaggressions are not appropriate. We agree to this conduct for any NeuroData sanctioned event/activity, including: hackathons, presentations, workshops, social gatherings, social media, or any other online media.
We agree to dress appropriately in lab spaces (both physical and virtual). Lab members are expected to be fully clothed in the work environment (including virtual meetings). Clothing with offensive or inappropriate designs or stamps should not be worn when interacting with lab members or representing the lab in any capacity. We also agree to avoid using profane language in public comments, code, or PRs.
(Much of this language is taken from hack code of conduct.)
Radical Honesty and Safety
We agree to speak with one another honestly (see below for a description of "impeccable agreements"), while at the same time maintaining psychological safety (to avoid crises) for all team members at all times. This means feeling comfortable both providing and receiving honest critical feedback. While "feeling comfortable" is not something that we can commit to, we agree to strive to communicate with one another in a fully open fashion.
Lab Spaces are Brave Spaces
We agree that lab spaces should embody the five tenets of brave spaces, as defined here:
- “Controversy with civility,” where varying opinions are accepted
- “Owning intentions and impacts,” in which students acknowledge and discuss instances where a dialogue has affected the emotional well-being of another person
- “Challenge by choice,” where students have an option to step in and out of challenging conversations
- “Respect,” where students show respect for one another’s basic personhood
- “No attacks,” where students agree not to intentionally inflict harm on one another
Lab members are free to profess any belief and debate any issue they choose provided they continuously demonstrate respect for each other. Disagreement is an inherently good thing as it can help us gain perspective for another point of view, examine our own beliefs and opinions, and look for common ground. Disagreement in a respectful environment is an opportunity for collective growth. These benefits are lost, however, when debates become attacks and we cease to show respect for each other. This also means that there is a responsibility on the offended party to make the offender aware of their offense if they would like change. We define a "challenging conversation" as being a conversation which includes topics which you choose not to engage in (all conversations (except PI mandated research conversations) in the lab are opt-in). People who choose to talk about challenging topics around you do not have priority in the workspace and you can always ask a group to leave. Respect implies that when someone asks you do something reasonable, like stop talking about a certain topic or to talk somewhere else, you respect them and their boundaries enough to listen. "No attacks" implies no malicious intent directed at anyone. Pointing out shortcomings via constructive criticism is possible to perceive as an attack, but we commit to assume the best of our peers. We should feel free to clarify with anyone, at any time, if a comment is intended as an attack or not. Clarifying intent is likely to alleviate harmful misconceptions. Nobody is perfect, and we will work with each other to make our work environment better.
Frequent Productive Feedback
We agree to provide our supervisors and our supervisees with open, reliable, insight into our experience of our work and work environment. To give a concrete example, we agree that if we feel unduely stressed for any reason (including non-work related reasons, such as having a difficult time sleeping one night), we agree to tell the supervisor the next day.
Before even starting a conversation, consider the other person's perspective and current state of mind. There are times that each of us is more or less receptive to feedback, and there is therefore a balance of finding the right time, but not waiting too long. About one day later seems about right for most things. Then, the general flow of the conversation would go like this:
- State the facts, e.g., "I failed to sleep"
- Speculate about the causes, e.g., "which I believe is largely because I don't believe I'll be ready with the deliverable by the agreed upon date".
- Inquire about the other side of the story, e.g., "what do you think is going on".
- End the meeting with a concrete change, e.g., a change in perspective or a change in behavior.
Some links that might be useful to understand best practices for giving and receiving feedback include
Feedback for jovo can be provided directly in the #jovofeedback slack channel, including anonymous feedback. One-on-one meetings with jovo can be scheduled through Calendly.
Contacting People Outside the Lab With Questions
We agree that contacting people outside the lab with questions is effectively asking them to spend their limited time helping us. Since we would prefer to be the ones offering help rather than receiving it, we agree, prior to asking them for help, to: 1. Consider what it would take for us to solve our problem (this means searching the literature, exploring and writing code, reading books, etc). 2. Consider what it would take for peers within the lab to help with our problem (this means asking labmates for help or literature recommendations, or asking jovo for help or literature recommendations). 3. Make a judgement call that the ratio of the effort required in #1 or #2 to the effort required to contact someone outside the lab is high enough to justify the potential cost to their time.
If we do have to ask people outside the lab questions, we agree to make sure to clarify to them the extent to which we are grateful for their time, and that we have exhausted our internal resources prior to reaching out. Any questions we ask should be about pointing us in the right direction, rather than asking for a time to speak with them.
Any day that JHU is open, unless otherwise specified, we agree to be physically at the lab (or active online if campus is closed and university is open). Our experience dictates that in person communication and happenstance meetings can dramatically increase our productivity. At the same time, many of us need some time to think or write in quiet, often in isolation. Therefore, we have found that a balance of the two is optimal for everyone (so far). The particular distribution of time alone vs. with the team is idiosyncratic per person. To respect the individual with the goals of the team we each agree:
- Work 40 hours per week, approximately 50 weeks per year, on our two primary research projects.
- Conduct at least half of one's daily work during typical hours (approximately 10am to 6pm Eastern Time, Monday to Friday on all days JHU is open).
- Respond to one another within one school day.
- Other hours/days: there is no expectation of responding to email/slack.
- When posting in channels, keep messages concise and professional.
- While people are in the lab, we will not intentially interrupt/disturb other people if their headphones are on (or they provide another indicator that they are working).
- Any questions that result in a conversation (rather than a simple answer) will not happen in the lab area, but rather, the lounge, in kavli, etc.
- If you aren't going to be around any given day, we agree to tell team and mark NeuroData calendar so the rest of the team knows.
- Up to 1 day per week working "remotely" (meaning from home when JHU campus is open, and offline when JHU campus is closed) will typically be acceptable with explicit permission, additional working from home days any given week must be justified.
Your most valuable resource while on this team is the team itself.
We agree that in the rare event that a supervisor requests your presence in the lab for a particular time/day, we will make our best efforts to be there, barring family responsibilities overruling research priorities. This includes: impending grants, conferences, or other internally/externally imposed deadline.
Regular Sub-Group Meetings
We agree that at regular intervals (no less frequent than every other week), each sub-group will meet in person (or via Zoom when campus is closed) for at least 30 minutes. The goals are:
- document past week's progress towards first author papers
- get feedback/mentorship from brilliant team of collaborators on existing bottlenecks
- make plan for next week
Every team member is expected to attend at least one of these regular meetings.
To facilitate achieving & documenting these goals, we each agree to the following for each meeting:
- Work 40 hrs on these goals, leaving time for other interesting stuff as it comes up.
- Document research artifacts from past week goals in the slides, research artifacts can be anything like screenshots, plots that are good enough for the meeting attendees to give feedback on
- Arrive on time
- Update slides for each first author publication you have worked on for the week.
- Listen and provide constructive feedback to your teammates as appropriate
Each week we ideally move "next week" to "last week", and color code each element on the basis of whether it was completed (green), partially completed (yellow), or not even started (red), or something else (blue).
Weekly Lab Meetings
So we all get a more broad view of team activities, we agree to have 1 hour for informal talks, Q&A, and lunch all weeks that JHU is open (including summers, unless otherwise specified). We agree to practice before getting in front of 10-20 people, because every minute you speak you have implicitly asked many other people to devote to you, so please be respectful of other people's time. Each of us will present about 2x / year, depending on the size of the team. Swapping dates with other lab members will be allowed whenever it is mutually agreed upon. New lab members will be added to the beginning of the queue when they join, and will be required to present their life story while wearing a funny hat. Team members can sign up for a slot here.
Annual Group Hangouts
To build community and deeper inter-personal connections, and to have fun, we agree to having annual activities like going to dinner as a team. These occasions will include a celebration of past successes, as well as a discussion of collective goals for the subsequent year. These occasions will be organized by trainees.
Semi-Annual No JoVo Meeting
We will meet at least 2x annual without jovo to discuss ways we can collectively improve the lab, and provide detailed and radically candid feedback.
We agree to annual reviews, to document goals and assess trajectory, by filling out the following questionnaire annually, and then scheduling a meeting with leadership to discuss. These meetings will happen in July. We will also sign this agreements document each year at this time.
Annual Review of Agreements
We agree to annually review and update this Agreements document at the end of June.
We agree to the contributing to the following research artifacts, depending on the "stage" of our careers, and our career objectives. We have different milestones and deliverables, but annual productivity is a crucial component of any career development.
- MSE thesis students and Research staff: submit a first author publication each year
- PhD students: submit a first author publication once per each of the last three years of the PhD
- Postdocs: submit a first author publication once per year
Each team member will present at >=1 conference per year to get feedback, network, and professional development. Any conferences one desires to attend is fine, conditional on
- applying to present your work via some mechanism (posters, talks, or boothing), and
- for conferences with proceedings, a "submission ready" draft is provided >= 1 month prior to submission deadline, or
- for conferences with posters, a "presentation ready" poster is provided >= 2 weeks prior to the event.
Each postdoc will apply for funding each year, targeting a solicitation >2 months in advance. Anybody using "extensive" commercial cloud computing (>$1,000/month) is expected to write a AWS Research credits grant at least annually for <$10,000. If your PI is writing a grant and request a figure/table/text/etc., it becomes the absolute top priority, including over and above impending journal/conference/workshop submission deadlines, unless otherwise agreed upon.
In general, the research priority, for any student, is to be working towards writing a paper within the year. Other priorities include preparing for conferences where you are presenting. And then your PI may have other more immediate responsibilities to attend to, and may ask you to produce a figure/result/etc. with some urgency. In general, prioritize what is most important, rather than most urgent. However, deadlines with dates are infrequent, and typically strict. In other words, if we have an impending deadline, agreeing to it means that all research associated with that product is the top priority. Thus, unless otherwise stated, if a PI asks you to produce x, interpret that as the most urgent and top priority.
Three months prior to departure, make a 'transition' plan, wherein the technical expertise you have acquired in the lab is transferred to at least one individual in the lab. This includes, - If you have written a numerical package, that the core components of it are integrated into another package, so maintanence is no longer dependent on you; - A plan for how to address completing papers that are not yet 'done' (where done means we've already sent proofs back). This includes identifying an indivudal who will be the point of contact for future correspondence with journals, etc. Authorship inclusion and order may change in this process, including elevating/relegating to/from first author, as deemed appropriate by the corresponding author.
Typically, the person actually submitting the paper is the first author. This means that sometimes, if we get feedback after an individual has left the lab, and that person decides to prioritize other work, the 2nd author may be responsible for resubmitting, and the author order may switch accordingly.
- The primary ways for a NeuroData family member to contact jovo will be scheduling 1-on-1 meetings on calendly or directly in the NeuroData calendar, or direct messages on Slack and text
- If someone needs to get feedback on a figure from jovo, they are encouraged to mention him in the respective subgroup channel for the paper. This allows for feedback from not only jovo, but other members of the subgroup.
- 1-on-1 meetings are highly encouraged as a way to get feedback, for questions about long-term/career guidance, etc.
- Adminstrative tasks or general questions can be asked during the "recess" time (3-3:30 daily). These kinds of questions are likely not urgent, so if someone misses a certain time, they will be able to ask the next day.
Other Kinds of Agreements
- Supporting everybody that meets these agreements. In the event it seems to me that you are not meeting these agreements, I will discuss with you and make a plan with you to get you back to meeting these agreements. This plan will have a timelines of at least one month, and up to three months. At the end of that experiment, we will collectively decide whether the agreements have been met. The decision will ideally be unambigous.
- Prioritizing your personal and professional success over everything else in my life after my family & career.
- Receive annual feedback from all members of the team with an open mind, as well as completing my annual feedback.
- Sharpening my saw regularly.
- Upon receiving a draft, it is the top priority, modulo more urgent paper/grant deadlines.
- Have check-in meeting (as needed if requested by NeuroData family member) to facilitate more long-term/career guidance, check alignment between personal and team goals in terms of quarterly, annual, and career goals, and re-align as appropriate.
- Seeking to maintain lab diversity writ large, including age, seniority, intellectual interests and background, sex, ethnicity, race, country of origin, ability, etc.
Videoconference technology can make it especially hard for deaf, hard of hearing, and/or non-native English speakers to understand speech. Therefore, we agree to use closed captioning technology when presenting our work virtually, whenever possible. Such technology is available on Google slides, and Microsoft Office 365. In order to make this practice maximally effective, we strive to:
- keep captioning on even during question and answer periods
- link audio output to input, so the captioning can pick up speech from the audience. This can be done by playing audio out loud through computer speakers, or by using software such as this.
While we do not necessarily agree to this, some evidence suggests that early afternoon breaks are beneficial, especially those that involve being outside, getting blood flowing, and switching cognitive contexts. Therefore, some of us will strive to take them on a daily basis.
Summer Community Outreach
There are a variety of summer mentorship opportunities in partnership with some Baltimore public education institution (e.g. state university, public high school) that can foster stronger connections with our surrounding community. Examples include:
- ICM Summer Internship with UMBC or Morgan State - website
- Mentor student from local college through JHMI Summer Internship Program - website
- Research experience for women in Baltimore City schools through WSE's WISE program - website
- Volunteering at CodeScholar, a summer bootcamp for Baltimore School System students - website
The definition of an Impeccable Agreement (our goal for all agreements)
On this team, we agree that all our agreements to be impeccable. When we make an agreement, it will have the following properties (adapted from impeccable agreements:
- Document somewhere (e.g., asana accessible to all stakeholders, including exactly the definition of done, by whom, by when, possibly why. SMART Goals are the most effective mechanism in my experience.
- Discuss aiming for a full-body yes:
- All parties to the agreement must be free to say no (though not necessarily without consequences).
- Full body yes is the goal, admittedly, the bar is high.
- Revise as soon as you realize you cannot meet all aspects of the original agreement.
- You can renegotiate the scope, the form, or by when.
- For example, in our academic goals, we revise after we meet, to reflect our new expectations.
- Take 100% responsibility for your part.
- If for whatever reason an agreement is not met, only take 100% responsibility.
- Do not take responsibility for other people’s actions.
- Make sure nobody else is on your critical path for any agreements that you make.
Recommended Reading on Mentorship and Imposter Syndrome
Mentors can anticipate that many of their mentees may experience imposter syndrome. This can be compounded by stereotype threat and experiences of discrimination that have happened in the past or their current environment (which should be addressed by the team). Mentors can support their mentees (and vice versa) by educating themselves on imposter syndrome’s causes, indications, and counteractive tools or practices. Mentors can thus learn to listen in a new way to their mentees and validate their experiences, to use positive and constructive feedback to affirm their value, to share their own uncertainties and learning process, and to ask if their mentees would like support in connecting with resources such as affinity networks for people with shared experiences or articles with tools for managing imposter syndrome.
The following resources are recommended to all lab members, especially to those who serve as mentors. They are also encouraged to seek additional resources to learn how to create an equitable and supportive lab environment.
- mentoring someone with imposter syndrome
- mitigating imposter syndrome
- the minority experience and imposter syndrome
Individual Commitments to Upholding Agreements
Certain agreements require a member of the lab to commit to be an organizer. Each year, a new member of the lab will commit for the following year or until someone else agrees to take responsibility. The following individuals are currently committed to upholding the listed aspects of the lab agreement.
- Weekly lab meeting: Ross
- Semi-annual social events: Ronan
- Semi-annual no-jovo meetings: Ali SE
- Annual lab trip: Sambit
- Getting diversity interns: Tommy