Research and Evaluation

Introduction

The Bayswater Institute way is to start where the client is. Each engagement is bespoke and sensitive to the context. Drawing on decades of experience in sociotechnical systems and action research we aim to provide understanding and learning that can support and guide change. We develop and implement approaches to evaluation that are rooted in the real world and that facilitate the change process. Recognising recurrent barriers due to technology, psychology and social factors we help to unlock progress and walk with you on the path to transformation.

Through combined experience in sociology, group dynamics, public policy, technology development, business development and user centered design we can provide:

Evaluation
Programme management and consultancy
Analysis (quantitative, qualitative and multi-method)
Grant application and project delivery.

Our Skills & Areas of Expertise

Leadership

Leadership is a constant and contentious issue across all sectors of public and private life. It gained prominence in the 1990s as Business Schools began to target cohorts of hopeful young managers and inculcated in them many of the prized attributes of leadership – often drawing inspiration from historic figures, from Agamemnon to Stalin, from Churchill to Genghis Khan.
At the BI we have a more social view of the psychology of the leader. One focus we have found useful is to focus in on potency of the leader. In this regard we have found it very helpful to understand the capacity of the leader to facilitate the growth and development of those being led. Issues abound here too but in line with strong facilitation comes observation of the BECM characteristics of the leader. The leaders Being, her manner of Engagement, how she Contextualises her legacy knowledge and how she Manages her engagement. These BECM qualities tell us a lot about the leader and her abilities to lead by virtue of her facilitation of her followers.

Bell, S. and Morse, S. 2011. Being, Engaging, Contextualizing and Managing Matrix—a Means for Nonspecialists to Assess Group Dynamics? Embedding Technology in Practice. Systems Research and Behavioural Science. DOI: 10.1002/sres.1088.

Double Task

Double Task is fundamental to BI and underpins the Wisdom in Groups residential. From its origins in the work of Harold Bridger to our various interpretations of its efficacy in agency, we see the evolution of a really useful theme – making people more reflective, learning people. We would go further and say, making people better people because of reflection.

Change requires collaboration. The Institute uses Double Task to develop our client’s capabilities to work in and understand groups of all kinds and levels. We call this capability GroupAware and recognise it as a significant contribution to personal development, efficiency and resilience.

The need for people to get better at the qualities which make them ‘stand out’ is becoming more apparent. In no area is this more apparent than in the realm of work and the threat of ‘the rise of the machines’. Some people have argued that Artificial Intelligence (AI) will make people redundant. We beg to disagree. In a machine-age it might be a good idea to maximise our human advantages – becoming better people. We should not attempt to compete with machines or be anxious about this. Our artefacts are not us.

Why is Double Task (DT) important in this regard? Well, human consciousness has irreducible differences from Artificial Intelligence and we can and should build up on our powers – DT helps us to do this. There remain many differences between computers and people:

Difference # 1: Brains are analogue; computers are digital

Difference # 2: The brain uses content-addressable memory – a kind of “built-in Google,”

Difference # 3: The brain is a massively parallel machine; computers are modular and serial.

Difference # 4: Processing speed is not fixed in the brain; there is no system clock.

Difference # 5 – Short-term memory is not like RAM – short-term memory holds “pointers” to long term memory whereas RAM holds data that is isomorphic to that being held in store.

Difference # 6: No hardware/software distinction can be made with respect to the brain or mind – the mind emerges directly from the brain, and changes in the mind are always accompanied by changes in the brain.

Difference # 7: Synapses are far more complex than electrical logic gates

Difference #8: Unlike computers, processing and memory are performed by the same components in the brain

Difference # 9: The brain is a self-organizing system – the brain is self-repairing something known as “trauma-induced plasticity” occurs after injury”.

Difference # 10: Brains have bodies – the brain “offloads” its memory requirements to the environment in which it exists.

Bonus: The brain is much, much bigger than any [current] computer1
Our brains provide a basis for our consciousness and if we focus on our:

Analogue capacity to identify and work in flows
Google like aptitude to associate
Capacity to parallel process
Ability to accelerate and respond to conditions
Systemic memory
Holism and how this impacts on our consciousness
Commensurate complexity in relation to a complex world (mirror)
Combinatorial facility to remember and think
Potent self-organisation
Sympathy to our embodied environment

Reflective practice is the key – and this if wonderfully enhanced by DT. In short, DT can help us to maximise our human strengths.
It is very easy for any of us to get ‘lost’ in the tumultuous callings of the world. We get stuck into our survival task and forget what it is that makes any of this matter. The Task behind the Task. The second Task. Interpretation of Tasks 1 and 2 are fluid but for our purposes here think of them as the explicit and the implicit. Double Task helps us to become potent integers in our environment and to make our aspirations materialise and matter.

Bell, S., Mahroum, S. and Yassin, N. (2016). Towards understanding problem structuring and groups with triple task methodology ‘e’. Journal of the Operational Research Society. doi:10.1057/s41274-016-0017-2

Bell, Simon, Berg, Tessa, and Morse, Stephen. (2016) Rich Pictures: Sustainable Development and Stakeholders – The Benefits of Content Analysis. Sust. Dev., 24: 136–148. doi: 10.1002/sd.1614.

Bell, S. and Morse, S. 2013. An Approach to Comparing External and Internal Methods for Analyzing Group Dynamic. Group Dynamics: theory, research and practice. 17, 4, 281 – 298.

Bell, S. and Morse, S. 2010. Triple Task Method: Systemic, Reflective Action Research. Systemic Practice and Action Research. DOI 10.1007/s11213-010-9171-7

Save

GroupAware

The Bayswater Institute provides a range of research, consultancy and professional development services to individuals, groups and organisations as they implement change in our complex, constantly shifting world.

Relentless focus on “getting the job done” leaves people alienated from the group and disengaged. This dilutes the common purpose, creates unnecessary barriers and results in sub-optimal outcomes. GroupAware provides a new way of seeing teams or groups and provides insights into more effective team working.

GroupAware empowers people, at all levels of organisations. This leads to improved team performance, deeper reflective practice and the resilience to catalyse change in a complex world.

Rich Pictures

Rich Pictures are un-crafted, naïve diagrams drawn without necessity for any skill or accomplishment which nevertheless provide profound insights into a given context.

In our adult lives we rarely make use of diagrams and, when we ask a group of people to draw the main elements of their context there is often an opportunity to draw out (educe) occult and hidden issues which have been the cause of pain, stuck-ness and conflict. The Rich Picture can be a therapeutic means which allows a group or community to begin to make progress and to heal.

Cecilia Agrell, Simon Bell, Filipo Bosco, Dermot Diamond, Jenny Emnéus, Anthony Guiseppi-Elie, Atkins Katusabe, Jim Lynch, Stephen Morse, Francis G. Moussy, Fionn Murtagh, P. K. R. Nair, Pamela J. Weathers .(2014) Transdisciplinary Sustainability: The Council for Frontiers of Knowledge. International Journal of Transdisciplinary Research. 7, 1, pp. 1-26

Bell, S. (2012). DPSIR = A Problem Structuring Method? An exploration from the ‘‘Imagine’’ approach. European Journal of Operational Research. 222, pp. 230 – 360.

Bell, S. and Faren Bradley, J, (2012). London’s Olympic legacy and the Imagine methodology. Local Economy, 27, 1, pp. 55 – 67. DOI: 10.1177/0269094211425325

Bell, S. and Morse, S. 2013. How People Use Rich Pictures to Help Them Think and Act. Systemic Practice and Action Research. 26, pp. 331 – 348.

Soft Systems Methods

Peter Checkland is the ‘father’ of Soft Systems Methodology. It was in 1981 that he first published: ‘Systems Thinking, Systems Practice’ where the notion of SSM was described in detail and the problem structuring community were given the necessary tools and techniques to think outside positivist and constructionist ‘boxes’ to engage with the systems which sit behind the systems of our worlds.

SSM at its best is a license to think, an opportunity to consider beyond the tramlines of our usual framings, a challenge to question assumptions and be creative in a collaborative manner. At worst it becomes another set of boxes to tick and ‘things to do’. Formalism and unimaginative conservatism are the enemies of any vibrant methodology.

At BI we apply SSM creatively and, in derivative methods like Imagine, with specific objectives in mind. Our aim is to be courageous and to offer clients unique vistas and not ‘more of the same’.

Bell, S. and Morse, S. 2013. How People Use Rich Pictures to Help Them Think and Act. Systemic Practice and Action Research. 26, pp. 331 – 348.

Bell, S. 2011. From Sustainable Community to Big Society: 10 years learning with the Imagine approach. International Research in Geographical and Environmental Education. 20, 3, pp 247 – 267.

Action Research

Action Research has a thousand incarnations but at BI we keep true to the fundamental tenant – to work with and not upon, to start from where the client ‘is’ and to facilitate processes of learning and not impose fixes (no matter how apparently benign). At the BI we operate as Action Researchers. We aim to be at one with the concerns and anxieties, worries and opportunities of our clients. From this ‘inside out’ perspective we can assist with change programmes and help the client to problem structure, to bring forth the genius within.

Eason K. D. Action learning across the decades: case studies in health and social care settings 1966 and 2016 Leadership in the Health Services, 30(2) 118-128
https://doi.org/10.1108/LHS-11-2016-0057

Hoare, Adam and Ken Eason. A Socio-Technical Approach to Evidence Generation in the Use of Video Conferencing in Care Delivery. IJSKD 6.2 (2014): 36-52. Web. 2 Nov. 2017. doi:10.4018/ijskd.2014040103

Report on the Project: Better Outcomes for People with Learning Disabilities – Transforming Care

Bell, S., Correa Pena, A., Prem, M. (2013). Imagine coastal sustainability. Ocean and Coastal Management, 83, pp. 39 – 51.

Mahroum, S., Bell, S., Al-Saleh, Y. and Yassin, N. (2016) Towards an Effective Multi-Stakeholder Consultation Process: Applying the Imagine Method in Context of Abu Dhabi’s Education Policy. Systemic Practice and Action Research. 29, pp. 335 – 353. DOI 10.1007/s11213-016-9367-6

Psychodynamics

At the BI we have a long legacy of and familiarity with and use of psychodynamics. The term has a variety of definitions but the one on Wikipedia is fine: “in its broadest sense, [psychodynamics] is an approach to psychology that emphasizes systematic study of the psychological forces that underlie human behaviour, feelings, and emotions and how they might relate to early experience. It is especially interested in the dynamic relations between conscious motivation and unconscious motivation”.

At the BI we try to never make the mistake of assuming that the nature of a complex reality is adequately summarised or presented by its apparent and obvious manifestation. Sometimes this may be true but rarely and it is not to be trusted. We will apply our unique psychodynamic methods to look beyond and behind the apparent; looking to engage with root causes.

Bell, S. and Morse, S. 2011. Being, Engaging, Contextualizing and Managing Matrix—a Means for Nonspecialists to Assess Group Dynamics? Embedding Technology in Practice. Systems Research and Behavioural Science. DOI: 10.1002/sres.1088.

Bell, S. and Morse, S. 2013. Towards an understanding of how policy making groups use indicators. Ecological Indicators. 35. pp. 13 – 23.

Bell, S. and Morse, S. 2013. Groups and facilitators within problem structuring processes. Journal of the Operational Research Society. 64, pp. 959 -972

Embedding Technology in Practice Read more ...

Embedding Technology in Practice

At the BI, we are dedicated to the use of theory in practice. We see the use of technology not only as a source of efficiency but as a way of improving the quality of people’s lives. However, we recognise that many large-scale technology projects fail to deliver on their potential benefits. We also recognise that often, people spend most of their time circumnavigating technology that is supposedly there to help. Hence, we see the role of theory not as an esoteric art of interpretation but as a fundamental method of understanding and learning from the successes and failures in the deployment of technology. Understanding the role and experience of using technology from all stakeholder’s points of view is central to our approach to unlocking the benefits. We are also committed to cumulative learning from efforts to embed technology into practice. We believe that the poor accumulation of evidence is due to an approach to evaluation that is to narrow and focused on success or failure rather than learning and knowledge creation.

Health and Social Care

The BI has decades (actually it is centuries when added up but that just makes us feel old) of experience in the development and use of technology. We have worked in many sectors of the economy including entertainment, manufacturing, services, Government and IT. We have worked all over the world including developing and developed countries. We bring an appreciation of different cultures that spans ethnicity to communities of practice. A lot of the work we have undertaken has been in the health and social care sector. This remains an area of particular interest to us but we recognise that many of the issues we address there are seen elsewhere as challenges.

A. Hoare, Factors Affecting the Move to an eSystems Approach to Remote Care Delivery. 2016 9th International Conference on Developments in eSystems Engineering (DeSE), Liverpool, 2016, pp. 7-12.
doi: 10.1109/DeSE.2016.3

Technology Development

Although we work a lot in the evaluation of technology projects we recognise that there is a reciprocal relationship between a technology solution, evolving in collaboration with practice, and the evaluation framework. We actually have experience starting and developing small companies with all of the challenges that includes. This means that we can work with clients and immediately start “from where they are” in terms of their journey.

Hoare A, Factors Affecting the Move to an eSystems Approach to Remote Care Delivery, Conference: 2016 9th International Conference on Developments in eSystems Engineering (DeSE)

Bell, S., Benatti, F., Edwards, N. R., Laney, R., Morse, D. R., Piccolo, L. and Zanetti, O. (2017) Smart Cities and M3: Rapid Research, Meaningful Metrics and Co-Design. Systemic Practice and Action Research. DOI 10.1007/s11213-017-9415-x

Bell, S. 2002. Surfing the Third Wave: Experiential Reflections on New Working Practices. Systemic Practice and Action Research. 15, 1, pp. 67 – 82

Commercialisation & Business Models

Commercialisation and Business Models

For many companies, the viability of their product or service is a matter of getting the business model right. The move to service based models and managed service approaches has gained favour over the straight hardware sale in recent years. This is based much more on a partnership model of purchasing and the development of ongoing customer relationship management. However, this model is at odds with many public-sector procurement approaches. This tends to drive innovative service development into the private business-to-consumer market rather than the public business-to-business market. This is one of the key reasons that the public sector fails to embrace and gain the benefits of innovative approaches. Hence, the approach to commercialisation and the business model is defined not just by the technology, product or service but by the way the market is intended to be accessed. We bring this broader understanding into our work with clients and it allows us to provide support for evaluation strategies that address the route to market and making the sales argument.

Realistic Evaluation

We live in a complex world that is evolving rapidly. The top-down approaches to product and service development favoured by bureaucracies and slow-moving industries are no longer fit for purpose. Bottom-up approaches that work with the front-line staff in terms of product and service development increase the chances of success but require a more iterative approach to development. This is why action research approaches are more appropriate for embedding technology into practice. This places evaluation in a different role to the classic pass/fail gateway. Evaluation now becomes part of the understanding and learning process of how the product or service is developing and meeting the needs of stakeholders. Further, different stakeholder groups have different requirements for “success.” The development of realistic evaluation shifts from a question of “does it work?” to “who does it work for, in what context and how?” Evaluation broadens out to use appropriate tools to consider these questions for stakeholders and to collate qualitative and quantitative evidence in support of the different points of view. We are constantly evolving our portfolio of tools and working with clients and partners to ensure we can support and facilitate this learning process.

Bell, S. and Morse, S. (2011). An analysis of the factors influencing the use of indicators in the European Union Local Environment. 16, 3, pp. 281 – 302.

Social Value / Impact

We are often involved in projects that have outcomes that would normally be considered to be out of scope. For example, some of our projects have resulted in reduced travel to hospitals for regular consultations or therapies, increased confidence, empowered and educated citizens and reduced anxiety. These outcomes are out of scope of many interventions because they are not linked to any economic value as far as the funder of the intervention is concerned. For example, a GP does not get paid any more for a patient who understands their condition or takes a more proactive role in their care. The reality is that patient will probably need less statutory care and may have reduced impact on the care system in later years. The impact of constraining any intervention to a measure of how it compares with the current way of doing things means that new ways of doing things often appear more expensive. This serves to lock payment into the current model and forms a significant barrier to reconfiguration. If there were economic proxies for outcomes such as increased independence we could at least begin to discuss outcomes more broadly. This is the role of social impact and social value. It is the beginnings of accounting for outcomes that are valuable to people but are not represented in our current economic thinking.

Bell, S. and Morse, S. 2011. Sustainable Development Indicators: The Tyranny of Methodology Revisited. Consilience. 6, 1, pp. 222 – 239.

Ergonomics

Ergonomics is about ensuring that the tools, the equipment, the environments etc that we create for people recognize the characteristics and needs of human beings and are thus ‘fit for purpose’. People are usually engaged in specific tasks and the design goal is to support how people engage in the specific work that they do. Physical ergonomics is the most widely recognized domain of the discipline: the physical design of controls and displays, the design of car seats for children etc. However, with the dawn of the digital age there is growing interest in cognitive ergonomics: developing software that recognizes how people search for information, store things in memory, solve problems etc.

One area of growing concern is human-machine interaction as the machine becomes more sophisticated and artificial intelligence means it can do more of the work itself. Will this render the human redundant? Will it mean the human monitors the work process most of the time and only intercedes when there is a problem? The evidence from many studies is that we are very poor at interceding in emergencies: we need to understand the emergent situation in detail before selecting the right action and that usually takes too long. Or are there ways of utilizing the general sense making, reflecting and adaptive capabilities of human beings.

Ken Eason, Afterword: The past, present and future of sociotechnical systems theory, In Applied Ergonomics, Volume 45, Issue 2, Part A, 2014, Pages 213-220, ISSN 0003-6870, https://doi.org/10.1016/j.apergo.2013.09.017.
(http://www.sciencedirect.com/science/article/pii/S000368701300210X)

Sociotechnical Systems

Sociotechnical systems theory (or socio-technical systems theory) began in the 1950s as a result of the Tavistock Institute of Human Relations in London studying mechanization in the weaving mills and in coal mines. The theory recognizes that work gets done by human resources (organized in social systems) and technical artifacts (organized in technical systems). A fundamental principle is ‘co-optimisation’: the technical and the social have to work together. And a fundamental finding, reported numerous times for over 60 years, is that we typically design technical systems without understanding how the social systems work and vice versa with the result that overall performance is often sub-optimal.

Sociotechnical systems theory is also an open systems theory. It recognizes that the work system has to be sufficiently adaptive and resilient to be able to cope with changes in the demands made on it: markets may change, government policies may change etc. So a system optimized for one day may be suboptimal the next. The theory has stimulated design approaches that recognize that it is the human resource that can recognize new demands on the system and find ways of changing the system so that it is resilient and flexible. This means organisational design solutions emphasize the need to give people local control over their work and particularly the technology they use and that a powerful way of achieving adaptability at work is to create semi-autonomous work groups.

The theory draws attention to the double task that every human resource in a work system needs to undertake. They are there to undertake the work of the day and they are also there to reflect on new challenges and opportunities and how the system needs to evolve to sustain high performance in the future. The work we do to help people in organisations reflect on what is possible and explore future ways forward is founded on these principles.

The introduction of technology into the workplace has always been associated with disruption. For many, their private lives have also become dominated by the use of technology. The impact of smartphones and social media is remarkable in terms of the speed of development and global adoption. The study of sociotechnical systems encompasses the sociological, psychological and technological factors in the interaction between human beings and technology. In an effort to understand what works and what can be achieved sociotechnical thinking looks at the various human and technological components as a system and explores the enablers and barriers to behaviour they provide. The approach also underpins approaches to project management that are less rooted in linear thinking and more related to co-creation and collaboration. Introducing concepts such as minimal critical specification as the start point of a technology that can be evolved into a solution that really embeds in daily practice. This leads into thinking about technology platforms and system reusability and repurposing. It seeks to explain why the private sectors is so much more adept at deploying and developing these approaches than the public sector.

Hoare A, Eason K, A Socio-Technical Approach to Evidence Generation in the Use of Video Conferencing in Care Delivery, International Journal of Sociotechnology and Knowledge Development, 6(2), 36-52, April-June 2014: 36-52. Web. 2 Nov. 2017. doi:10.4018/ijskd.2014040103

Bell, S. and Walker, S. 2011. Futurescaping Infinite Bandwidth, Zero Latency. Futures. doi:10.1016/j.futures.2011.01.011

Bell, S. and Wood-Harper, A. 2014. The Innovation of Multiview 3 for Development Professionals. Electronic Journal of Information Systems for Developing Countries. 63, 3, pp. 1 – 25.

Productivity

Too often productivity is reduced to a simple ratio of how economic inputs can be leveraged to produce economic outputs. This can be at the macro level with talk of GDP growth or the micro level with analysis such as return on investment and profit growth. The convenient blind eye turned to ‘externalities’ and inability to incorporate social value as part of the outcomes ensures that the story is often ‘half-told.’ We see this most strikingly in sociotechncial systems where the technology alone is seen as the bringer of productivity gains. We often provide summative assessments of projects to introduce technology ‘in pursuit of productivity gains’ where many factors we could have predicted would impact on the outcomes have been ignored. These include:

A linear approach to the project where all of the understanding is assumed up-front with no capacity to learn,

Poor engagement with all stakeholders to explore assumptions and their view of outcomes,

Comprehensive systems analysis of the intervention,

Embedding of the technology into practice including: training, maintenance, support and change management,

Impacts of privacy, data governance and sharing, interoperability with other systems and organisations,

Clarity of costing between owning infrastructure and using managed services,

Plus many more factors.

We therefore believe that any discussion about productivity gains does not start with a focus on technology or process but with clarity of outcomes (economic and social) of what is being proposed. Our experience indicates that resources invested up-front in understanding the intent can optimise value for money in the longer term.

Wisdom in Groups

GroupAware empowers people, at all levels of organisations. This leads to improved team performance, deeper reflective practice and the resilience to catalyse change in a complex world.

Imagine

The Imagine approach is intended to assist organisations and communities of all kinds to improve their long-term sustainability.