#92 the power of the performance metric – getting your jurisdiction back on track - huff
Upload: project-for-public-spaces-amp-national-center-for-biking-and-walking
Post on 29-Nov-2014
538 views
DESCRIPTION
TRANSCRIPT
The Power of the Performance Metric Getting Your Jurisdiction Back on Track
Margot Ocañas Herbie Huff Madeline Brozen Chanda Singh
Pro Walk Pro Bike 2012 Thursday, 9/13/12
Presenter
Presentation Notes
Margot Good morning – thanks for coming, this is “The Power of the Performance Metric” qualify the audience – qualify how they are engaged in pedestrian-ism – who are the advocates and who are not – each can apply from different pressure points. any good active transportation jiokes?
Today’s Presentation
• Background • Rationale: Why? What? • The Team • Metric Prioritization: The Process • The “WIP” Deliverable • Get Started!
Presenter
Presentation Notes
Margot What are our goals with today’s presentation? A bit of inspiration or a bit of motivation. The goals is to provide you with the real world/ behind the scenes of a metric exercise – that it can be done with limited resources and data - it takes creatively and different professional perspectives. Do not wait, this initial exercise can open the discussion to other areas. We felt there was value in detailing the back and forth of our discussions, so the focus here today is on the process and the presetnation of a living document. We are very comfortable calling out materials draft, and are not fully polished We want to explain how we came up with performance metrics for City of LA so that others can do the same in their own jurisdictions -The primary purpose of our presentation today is we’ve tried to extract lessons learned from our experience We want today to be interactive – for you to take something home with you that can start next steps And we want to learn from you, we think of this project as a work in progress Here’s an outline of what we’ll go over today. I’ll give you some background on the project, and what need we think it fulfills. Then my colleagues will review how we chose metrics, our final product, next steps for you all to get started with a project like this, and next steps in the City of LA
Background
• Health and Built Environment • CPPW / RENEW LA • Complete Streets Prototypes • Challenges
The City of LA Context:
• WHY performance metrics? • WHAT need do they fulfill?
Presenter
Presentation Notes
Margot More and more, public health is coming to the table when thinking about the built environment – their participation at the table really crystallized the the now irreffultable nexus between health and the built environment. We’re seeing links, and more research is confirming this is the case, that our built environment serves as a health determinant. In particular, we see that the built environment can have great influence on how much exercise we get, which plays a big role in many of the chronic diseases we are trying to deal with = such as diabetes, obesity, etc. In 2010, the LA County Dpt of Public Health received a Communities Putting Prevention to Work grant from the Centers for Disease Control. The purpose of this grant was to adopt and implement policies that prevent obesity. DPH granted three Los Angeles based organizations – Pacoima Beautiful, Green LA, and the City of Los Angeles These initiatives varied in size and scale, but all were to increase physical activity Realized that we were running into the same issues (describe issues / roadblocks) There seemed to be a fundamental lack of vision in LA – and that meant we couldn’t hold anyone accountable In addition, we were inspired by other agencies – new york, san francisco, and great work at some of less large jurisdictions, but also recognizing our climate and set up, we need to get a”fire lit” Figured – what could help all of our grantees? Why could metrics help you? What did we think their stated purpose / objective would be? That in the vacuum of a city mandate or centralizing agenda around mobility/sustainability/quality of life – numbers tell a story what’s not counted doesn’t count – inaction
The Team
• RENEW LA (DPH) • Technical assistance coordinators
– Ryan Snyder Associates – Complete Streets Program, UCLA
• Grantees – Green LA Coalition – Pacoima Beautiful
• Community partners – LACBC – Community Health Councils
Presenter
Presentation Notes
Margot So in order to get started, we brought together a diverse group of stakeholders Really important to get those voices in the room that can help challenge and discuss Also, rely on these folks for feedback, collecting some of this data,
ACTIVITY TIME!
What is your vision? Example: Health, equity, access to open space Who are your partners? Example: Bike Coalition Why? Because they do bike counts and know the bike data
Presenter
Presentation Notes
Margot – Our first activity will get us all thinking about first steps in undertaking a project like this Who are your partners? What is your vision for your jurisdiction or community? OR WHERE IS IT LOCATED? Can’t emphasize enough that this really sets the stage for the future metrics Go ahead and introduce yourself to the person sitting next to you and discuss. Then we’ll share some visions as a group. Let’s take about 5 to 10 minutes to brainstorm Worksheet – should have column for partners, what could their key function be, and visions – include examples
The Parameters
• How do you choose a metric? • What are the intended and unintended
consequences?
Presenter
Presentation Notes
Herbie I am going to try to open up some of the challenges of this exercise by talking through a few example metrics. I want to show you how we struggled to choose a metric, and the process we went through – of identifying a metric or goal or data set, then figuring out how best to formulate it in the report. Here are some common themes that will come out of the examples. I’ll elaborate more on these as we go. First, the choice of metric is a function of your vision, and what the folks at your table want to pay attention to. For us, that was biking, walking, transit… equity, open space, safety… these concerns led us to examine certain metrics. It’s also a function of what’s measurable, and I’ll talk about that in a second. Second, the choice of metric is a function of what data is available. Crash data, for example, we know is not a perfect indicator of safety problems or the lack therof. But there is a really robust and easily accessible database of crashes and we do want the city to make good use of it. Similarly, we have demographic and socioeconomic data from the Census, so we use a lot of that. We made a lot of overlays with Census data. There are other things – like, just to pick one out of the air – the spatial distribution of economic transactions – that we would love to do similar kinds of overlays with, but that data isn’t as readily accessible. Third, the choice of metric is based on the policy implications. You want to choose a metric that motivates action. Sometimes little details in how you formulate the metric affect the actions it motivates. I’ll explain more when I talk about parks.
How did we organize our thoughts?
• Metric – WHAT are we measuring? • Data Elements – HOW do we measure it? • Objective – What’s the PURPOSE of measuring it? • Policy – What ACTIONS are implied?
Presenter
Presentation Notes
Herbie So, before I leap into the examples let me show you the architecture we put in place to hold our brainstorming. This is how we generated metrics. Now this framework is not a science. I am sure that people in this room use the four terms up here in different ways than we do. We didn’t trouble ourselves with being terribly precise about these boxes. You can quibble endlessly about that and I’m sure that if anyone in here picked up our report they would see things they wanted to shuffle around between categories. I think one of the key lessons we want to convey is that you don’t really gain much from doing that, because precision is really hard to achieve. Just choose a way to be organized. This is the way we picked, so that we and our stakeholders and partners could generate ideas from a lot of different directions. 4 key categories we could think of – metric, data element, objective, and implied policy Sometimes we came with a policy first, and thought about what data or metric could support it later. Other times we came up with some data first that would be easy to obtain and figured out a potential metric later
The Brainstorm – Trial and Error
Presenter
Presentation Notes
Herbie Here’s kind of a visual of that brainstorm. And again the main thing I want you to get out of this is that by putting some kind of structure or architecture in place, it helps you generate ideas. This is all very complex and it’s easy to get stuck. The boxes also help you tackle things one at a time.
The Brainstorm – Parks
Presenter
Presentation Notes
Herbie Example: Our vision is more parks First thought for a potential metric was “acreage of parks”… but then we think about potentially how this would be used to inform action Current conditions in the city – there are actually huge parks, the problem is access / distribution throughout the City – in other words, we have a lot of acreage of parks total, but only a certain percentage of our City has easy access to that park Ultimately decided on final metric of “percentage of population living within a quarter mile of a park” Policy Implication would be add more small parks in park poor areas… and we know we like that policy
Existing Metrics
Level of Service
Presenter
Presentation Notes
Herbie Now I want to talk a little bit about LOS and how that shaped this project. What we saw in the City was that LOS was this very concrete thing that was being measured all the time, and that was driving policy. In part, LOS was the genesis for this whole project. We had a vision of the City that not only looked at other MODES besides cars but at all these other aspects as well – health, equity, sustainability. I might be preaching to the choir here, but what happens when the only performance indicator is related to cars? (i.e. volume to capacity ratio, or delay at intersections). You get decision makers and policy makers focused on moving cars, to the detriment of everything else. What’s counted, counts. I also want to make the comment that we chose NOT to include multimodal LOS in our performance indicators. Number one, we don’t have the capacity to calculate it. But number two, we chose not even to point towards it as a solution. Why? It has to do with what is and isn’t measurable, as I mentioned before. LOS measures a real, one dimensional thing. Delay. The number of seconds of delay is a real thing that can be measured and has real meaning. Now when you look at other modes, often you want to know about the quality of a bicyclists’ experience, the quality of a pedestrian or transit user’s experience, aside from just travel time or delay. This is what multimodal LOS is designed to do. But mathematically – and I’m a mathematician by training, you cannot collapse a multidimensional experience into one dimension without losing a lot of information. And often what you’re left with – and I think people who have experience with MMLOS know this – is a metric or indicator that is no longer sensitive to the things you want it to be sensitive to. So ultimately we decided that MMLOS wasn’t a great tool for our purposes. I think there is often an urge to use MMLOS, but that the real problem with LOS is all the procedures and policies that hinge on it. The real problem is with how we use LOS, not with LOS itself. And part of what our project does is blow the whole thing up and show the enormous gamut of things decision makers could be looking at instead of JUST LOS.
What data is available?
Presenter
Presentation Notes
Herbie Another example is the quality of the walking environment is something we definitely wanted to include But, we found a fundamental lack of local information We dug up what we could and highlighted its importance, but it’s very difficult to get more detailed data on this. We had to accept that we couldn’t make this item as obviously actionable as the other ones…. But we still include this information in our report. Lesson learned is use what you can get. Other cities may have more detailed inventories about the walking environment, but City of LA does not. – We included other data related to being a pedestrian, like crashes, ped counts, transit boardings, NHTS mode split data.
Existing Metrics
Vehicle Miles Traveled
Presenter
Presentation Notes
Herbie One of our stated goals as a group is the desire to grow the city in population without growing vehicle miles traveled Have candidate metric of VMT traveled on US roads, not very good local measures Realized this doesn’t really work well because its hard to interpret There are larger economic factors that influence VMT, data isn’t right geography. We’re going for actionable metrics. A decision maker can’t tell by looking at this if what they’re doing is working. We want metrics where you can see a call to action or the effect of action. In the end, we left this graph out and left the policy goal in.
ACTIVITY TIME!
Think of an objective and a corresponding metric or two. Example objective: Most short trips made by walking and biking Example metric: Mode share split for trips under 3 miles How would a decision maker interpret this? How does the metric lead to action?
Presenter
Presentation Notes
Herbie So now that we have a stated vision from our last worksheet – think of an objective. Or policy goal as the case may be. What are potential metrics that follow from a certain objective? For example, let’s assume most people in the room’s vision have to do with creating more walkable and bikeable communities An objective related to this vision could be to increase funding for walking and biking Then there are some potential metrics for this – one could be to track how much is spent on walking and biking, or how much is spent on walking/biking compared to walking/biking mode share Work with a partner the same one or another - to come up with a metric or two for as many objectives as you’d like. Does your metric seem to motivate action? What actions does it motivate? Are there unintended consequences? 5 to 10 minutes
Report organization
Transportation
•Metric / What? •Data elements / How? •Objective / The Goal? •Policy implication / Why?
Safety
Economics
Environment
Presenter
Presentation Notes
Participants have just finished beginning to think about what metrics they would choose and what are some of the issues. Now we’ll show them the end product of our own attempt to do the same. Our report is very much based on the audience we were trying to target. We wanted it to be something we can hand to a staffer. Maddie Structure of the report, I like the what how why goal, why as a less jargon way to explain our table. We had many discussions and iterations of our potential final product Decided finally on a report that could be handed to a decision maker – whether it be city staff person or councilmember, since many of us involved in project had these connections Because we wanted to highlight equity and health, these were key themes we used throughout the report
Q: How many people live within a quarter mile of frequent and reliable transit service?
• How: – Frequent and reliable: at least every 15 minutes
during the daytime + “15-minute map” – Census block population
• Goal: – 75% of new development near transit
• Why: – Sustainable growth, feasible option, physical activity
requirements, biking and walking connections
Presenter
Presentation Notes
Maddie Each line will come in on a click Here’s an example from transportation section of the report.
54 % of people do
not live within a 15-minute walk of frequent and reliable transit service
Presenter
Presentation Notes
Maddie
Transit service and income
23 % of people in low in areas do not live within a 15-minute walk of frequent and reliable transit service
Presenter
Presentation Notes
Maddie
Transit Service, Income, Geographic Disparity
62%
80%
91%
38%
20%
9%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Population Pop. Near FrequentTransit
Frequent Transit Stops
LA Basin
Valley
80 % of people in the Valley do not live within a 15-minute walk of frequent and reliable transit service
Presenter
Presentation Notes
Maddie
Presenter
Presentation Notes
Maddie And here’s page of our report Wanted to create pithy statements that were memorable, recitable, and made our point
Q1: How many injuries and fatalities happen to cyclists and pedestrians?
• How: – SafeTREC TIMS – Census and NHTS modal splits
• Goal: – Reduce fatalities and injuries by mode,
population, travel, etc.
• Why: – Create a safer environment for biking and walking
Q2: How does this distribution relate to how we travel?
Presenter
Presentation Notes
Maddie - Each line will come in on a click
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0
25
50
75
100
125
150
Car Bike Walk
Perc
enta
ge o
f Mod
al S
plit
Num
ber o
f Fat
aliti
es
2009 Fatalities by Mode for the City of Los Angeles
Fatalities
Commute Trips
All Trips*
*all trip data for Los Angeles MSA
Presenter
Presentation Notes
Maddie - Each line will come in on a click
64.3%
82.7%
94.1%
86.1%
33.5%
10.3%
3.7%
12.0%
2.2%
7.0%
0.9%
1.1%
1.2%
0.8%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Fatalities
Collisions
CommuteTrips
All Trips
Motor Vehicles*
Walk
Bicycle
Other
Presenter
Presentation Notes
Maddie -
Presenter
Presentation Notes
Maddie So now chanda will talk about how you take this process and make it your own
Common Data Sources • Demographics
– Block level census data • Travel behavior
– National Household Travel Survey (2009)
– Ridership data (from transit agency )
– Bike / ped counts from advocacy group
• Safety – Hit and run from PD – TIMS – SWITRS
• Health – 2007 Indicators for Health
Report (DPH) • Other
– Center for Neighborhood Technology
– City of Pasadena and Metro staff and reports
– Trust for Public Land City Park facts
– Intersection traffic volumes
Presenter
Presentation Notes
Chanda - Thanks Maddie - Now that we’ve gone over the purpose of this exercise, some of the challenges and thinking behind our metrics, and our report structure, I want to talk about one of the key parts to this whole process – and that is finding data! This was definitely one of the most challenging part of the process We knew about some data, but had to dig around for other types The key point I want to stress here is that it may be difficult to find what you have brainstormed to be the ideal type of data, at the right level of detail, etc. I think we’ve already made this point a bit already, but its worth reiterating that part of this exercise is leveraging what you already have And the good news is you don’t have to start from scratch Here is a list of some some data sources – I won’t read the list – that we used for our project. We have the full list in our report Census block and tract level data are key points, the National Household Travel Survey data, ridership data, crash data, traffic volumes, etc. Another thing to remember, is you’ll need your partners and established relationships to get the most out of this process For example, we had to rely and request information from our transit agency, local bicycle coalition, and others in order to get to our final product So the partners we brought to the table, as well as our other contacts really helped
Other places to look
• Regional data • Schools
– Fitness data – Mode share
• Developers – Traffic volumes – Turning movements – Speeds – Sidewalk inventory
Presenter
Presentation Notes
Chanda Here are some additional sources of data that we did not employ for our metrics product County or regional data School data such as fitness, or mode share Developers are also often required to collect data as part of new projects like speed, turning movements, sidewalk inventory, and others. These are examples of data that we did not leverage for this project, but we certainly could have if we had more time, and may choose to do so in the future So on that note, let’s discuss
DISCUSSION TIME! What other data sources are out there?
Presenter
Presentation Notes
Chanda What are other potential data sources – local, regional, state, national – anything that you think could be useful to consider?
Let your data tell the story
Presenter
Presentation Notes
Chanda - Now I want to share some lessons learned about visualizing data in order to get a powerful message Our entire process, as you may have gathered so far, was a lot of trial and error. Trying things and figuring out what spoke to us and others. A key lesson learned from our process is that almost anything spatial, you can overlay with population and get a measure of access, and often, equity As part of this, it’s important to make and state some pretty defensible assumptions For example, in order to measure access, we made assumptions about how population is distributed within a tract. When the census data states that 100 people live in tract, they could all live in one portion of the tract, but we made the assumption that they are evenly distributed within the tract in order to make statements like “30% of the the population lives within a ¼ mile of xyz” Just a note on census data – you can get population and population only at the block level, but things like demographics, are only available at the tract level – meaning this is as fine as you can get in detail Another lesson is to think about logical buffers to use when interpreting data For example, the California Air Resources Board recommends that sensitive use not be located within 500 feet of a major roadway or freeway due to health impacts related to motor vehicle particulate matter, so we used this buffer in our analysis Another example is the ¼ mile to 1 mile radius around certain features. The literature tends to use these values as reasonable walking or bicycling distance, so we used this when looking at access – for example, x% of the population does not live within a ¼ mile of a designed bikeway or transit stop For us, we had always envisioned our audience to be our decision makers – council members, commission members, or even senior staff, so one thing we did was to break our data up geographically that match political processes, so by council district, or community plan areas. We felt that these would be key attention grabbers. Overlaying spatial data with very common demographics data available from the census helped us tell a story about disparities in our City Participants will be able to present data in clear ways with understanding of some of the principles of overlaying spatial data with commonly available demographics data from the Census
Selling your product
Presenter
Presentation Notes
Chanda – why is it palatable? So now let’s just say, you’ve done the work – you’ve got this great product that gives new meaning to a lot of the data we already collect, why should anyone care? Some of these might be pretty obvious already, but I think they are worth repeating if you’ve already heard them We see our report as creating opportunities for decision makers – we are showing them that they have the opportunity to change something, and then see that change. For example, with our parks metric Herbie referenced earlier, I think our City already knew that the problem was the disparities in how parks were distributed throughout the City, but now not only are we showing them it is a problem, but if they do something about it, they’ll see a change in that map – they’ll see a change in those numbers, and that can be their legacy. We’re giving them the ability to look at the situation, choose to intervene, and then be able to say “I made that happen” – not for everything, but for many of them I think we all know – whether you work for a jurisdiction or not – that everyone is strapped for time and money. Its not that our jurisdiction staff are opposed to doing this kind of work, or thinking about these ideas, but they often do not have the luxury of doing so – they don’t necessarily have the time to be at the table and brainstorm how we can change the direction of the City So basically what we’re doing is doing a first draft pass at perhaps taking some of the work off their shoulders – we’re setting up the framework, culling data sources, and hopefully helping them prioritize projects – so in many ways, we are using the argument of “hey we did this for you, could you just take a look?” And that we did this with limited time and budget – this isn’t something that we wanted to be an onerous process – for ourselves, and hopefully for staff The other thing, is this document is not confrontational. We are simply pointing out facts. I think this is something we all do in our work, but it really helps it to have it one central place – it’s basically an assessment of our city’s health – and metrics are not a new idea in anyway If you go through this exercise through building a partnership like we did, then you already have some respected organizations who are interested in also helping you sell your product
What happens next?
Presenter
Presentation Notes
Chanda So I want to bring us back to our story for a moment We are still in the place of seeing what happens next with our project. This project arose from the fact that we felt that there was a lack of “evidence based” practice in Los Angeles, we were coming from a place where we saw other cities using metrics like ours as a dashboard – and taking a very rational approach to looking at how a city functions We all know how complex it is, but what this document has proven to be so far is a conversation starter that takes some of the complexity out, and that’s what it has always intended to be You never know who will end up in what position, and what will happen, but so far – we now have this in the hands of LA’s first ever pedestrian coordinator, Margot As part of a recent movement in LA, we are now hopefully working on a new health chapter to our General Plan, and I’ve been able to hand this over at our internal meetings as a key starting point, and the City wants this – staff sees this as work they would have had to do anyway, so we have basically eased up their work load An analogy we sort of made up here is the idea of getting a checkup. Ideally, ever year, we would all get a check up, and you’re not sure what you’re going to see. Maybe they’ll see you have high blood pressure, and they recommend you walk a little more. But if you never go, you get to a state where you can get very sick – or you’re responding to an emergency. Worse still, maybe you’ll think the problem is something else, so you are trying to cure a disease you don’t have. This is how we feel, and especially as non-city staff –that our city is sick, and our decision makers are operating in a void. They don’t even know how sick our city is. So this is analysis showcases some of those issues, and is transparent, and clear.
Kick-starting the effort
• Don’t wait – go for the low hanging fruit • Give yourself a time and/or budget limit • Learn from your peers • Intersection of research and practice – find
yourself some interns! • Figure out what your jurisdiction is tracking
already
Presenter
Presentation Notes
Chanda Participants will be able to list some readily available performance metrics that could be examined in their jurisdiction - So I just want to end on the fact that anyone can begin this process – and it’s just another tool in our toolbox to make change happen Don’t wait – go for the low hanging fruit. Hopefully you’ve gotten some ideas from our presentation today about metrics that speak to your vision, ways to visualize data, and some easy data sources. You don’t have to be a GIS expert to get some powerful data, you don’t have to be a total math whiz either. Learn from your peers! We don’t have to reinvent the wheel, and we definitely borrowed from resources we found just googling around One fun thing about this project is that this is really at the intersection of research and practice, so find yourself some interns! We had an amazing graduate student researcher, Rachel Cushing who has been floating around the conference, who really helped move our process along and did a bulk of the work. There are lots of students out there who want to work on interesting projects like these, and be part of discussions strategizing, so definitely partner with a local institution And lastly, spend some time digging around to figure out what your jurisdiction is already tracking, but perhaps just not using. That is definitely the case in our city – there is a wealth of information out there we didn’t end up using, but often there are just treasure troves of data not being used, so do some cold calling and you will eventually be connected to someone that can help. In a minute we’ll open it up, and we’d love to hear how your jurisdictions are tracking indicators, who you’re using information to inform city policy, and how do you make it available to the public if you do at all?
http://lewis.ucla.edu/content/completestreets-publications
Margot Ocañas Pedestrian Coordinator, LADOT Formerly, LA County Public Health Department [email protected]
Herbie Huff Transportation Planner, Ryan Snyder Associates
[email protected] Madeline Brozen Program Manager, UCLA Complete Streets Initiative [email protected]
Chanda Singh Policy Analyst, LA County Department of Public Health
Formerly, Ryan Snyder Associates [email protected]
Presenter
Presentation Notes
Chanda Thanks – here’s our contact – let’s open it up!