This section describes the last core construct of Secure Natural Communications – the intelligent network architecture that can fulfill and provision the user’s intentions. After the decode engine provided the interpretation of the user’s intentions (and assuming that the core infrastructure has enough capacity to perform these intentions), what remains is for the infrastructure to securely identify certain resources and assign them to the tasks at hand. Many of these issues are standard for designing any network and were addressed in some form for the Internet and Web. However the scale of the set of resources involved results in different requirements. Hence these topics need to be revisited.
Sample information intelligence research issues.
• Naming and addressing. This is the first requirement of any infrastructure design. How many elements will need to be addressed (think about all of the sensors, receptors, displays, etc.)? The number is easily in the billions. To support group communication, how will we deal with the exponential number of possible groups? Structure of the address space? Support for mobility?
• Location services. Whether via directories or search, the infrastructure must have a method whereby participants in secure natural communications can find each other.
• Application programming interfaces. Secure natural communications must provide a standard means for people to get access to its capabilities. These are standard mechanisms to request capacity, share documents, connect to people, apply privacy and security controls, and manage resources, reflected as a set of Application Programming Interfaces.
• Security and privacy. As the infrastructure provides more services it also learns more information about people. With more personal information on-line there is greater opportunity for misuse. Social media provides exposures. These issues are additive to today’s unsolved core security problems with the Internet. As we imagine the future infrastructure, we must build in security mechanisms. These include greater use of encryption, privacy hierarchies, and placing greater controls on anonymity – to help discover mischief.
• Group management. Group communication is popular and infrastructure support is required. First group communications was popularized with shared workspace solutions – notably Microsoft’s Sharepoint. Blogs, twitter, Facebook are examples of social media which require support for groups to communicate in an ad hoc fashion, share information, and/or multicast. There are discussion groups on the Internet and teams of people from many companies collaborating on a single, brief business transaction. Core infrastructure management must provide support for addressing and locating these groups; mechanisms to track their discussion threads; facilities for people to join and leave and learn the history of the group, and lightweight but adequate controls for security and privacy.
• Resource management. The role of an operating system is to manage resources in a single computer. The role of the traditional Network Operations System was to manage all resources in a telephone network. Today, the same infrastructure manages both data (IT) resources as well as telecommunications resources. Moreover, the fact that workloads are more portable - due to cloud computing, virtualization, and virtual appliances - implies that the traditional boundaries between management in a computer (Operating System) and management in a network (Network Operations System) is blurred. Hence, secure natural communications, by fusing together all resources and all applications must develop a universal approach to manage all resources. These resources must be supported across a diversity of operating environments, security needs, virtualization, and leveraging both public and private resources.
Wednesday, April 7, 2010
Thursday, March 25, 2010
Core infrastructure for secure natural communications
As opposed to the other key constructs, designing the core infrastructure is not a fundamental research issue but rather requires effective systems engineering. Given the design parameters that can be foreseen for secure natural communications – what is the best network design, how much capacity is needed, where to place concentrators, etc?
Today’s technology suggests the following general design about the core infrastructure. We imagine a world which is rich in sensors for both input (e.g. camera, speech) and output (e.g. displays). These are small and cost-effective hence they can be ubiquitous. They are in every room in every building and pervasive in outdoor malls and even in roadways. They are equipped with GPS so we know location and the sensors can communicate.
To achieve the natural communications and to connect these pervasive sensors required untethered devices with extensive coverage by wireless networks. But wireless cannot solve it all. Since maximum bandwidth is achieved through fiber optics we assume an extensive high bandwidth fiber backbone.
Key systems engineering issues
• Input sensor design. We imagine that people speak naturally and have their speech, and ultimately their gestures sensed by the infrastructure and operated upon. How will the physical sensors be designed? In order to get reasonable noise-free reception – do they need to be worn on clothing very close to the speaker? Can those be made natural enough, sufficiently low-cost, and powerful enough in a small footprint? Or will rooms be designed with acoustic systems and cameras to pick up on sounds and motion?
• Output sensor design. Players not only emit bits, but they also receive them. What is the overall design of the output system? Are there screens everywhere that people gravitate to? How much is visual and how much is spoken? What is the trade-off between personal devices and devices that are built into the infrastructure?
• Local wireless network. Today there are wireless networks with different design points for the home, office, mall, and airport. If we had a uniform assumption about the desirable network infrastructure: a high bandwidth fiber backbone and ample wireless networks to reach into that backbone, what would the wireless piece look like? What would it cost to configure? Do we have the right protocols designed for that set of assumptions? Spectrum?
• Spectrum. How should spectrum be allocated to balance all of the wireless needs between local area, metropolitan, and wide area?
• Wide area backbone network design. Similar to the wireless network observation. We have fiber networks today. Let’s create a specification to understand the needs of tomorrow. What would represent sufficient bandwidth for foreseeable needs? How would we build such a network? Cost?
While core network design is principally about systems engineering - improvement in core communications research technology will also improve the core network. Here are some of these core research areas motivated by the desire to create a core infrastructure for secure natural communications.
Sample core infrastructure research issues
• Fiber componentry. Getting more bits per second through a fiber link.
• Fiber network architecture. Designing the layout of fiber multiplexors and processors to improve network bandwidth
• Nanotechnology. Continued miniaturization is critical for sensor technology
• Wireless contention algorithms. Better signal processing to get more bandwidth out of limited spectrum
• Multi-media. How to utilize network bandwidth to address quality of service needs for disparate traffic patterns such as voice, data, and video.
Today’s technology suggests the following general design about the core infrastructure. We imagine a world which is rich in sensors for both input (e.g. camera, speech) and output (e.g. displays). These are small and cost-effective hence they can be ubiquitous. They are in every room in every building and pervasive in outdoor malls and even in roadways. They are equipped with GPS so we know location and the sensors can communicate.
To achieve the natural communications and to connect these pervasive sensors required untethered devices with extensive coverage by wireless networks. But wireless cannot solve it all. Since maximum bandwidth is achieved through fiber optics we assume an extensive high bandwidth fiber backbone.
Key systems engineering issues
• Input sensor design. We imagine that people speak naturally and have their speech, and ultimately their gestures sensed by the infrastructure and operated upon. How will the physical sensors be designed? In order to get reasonable noise-free reception – do they need to be worn on clothing very close to the speaker? Can those be made natural enough, sufficiently low-cost, and powerful enough in a small footprint? Or will rooms be designed with acoustic systems and cameras to pick up on sounds and motion?
• Output sensor design. Players not only emit bits, but they also receive them. What is the overall design of the output system? Are there screens everywhere that people gravitate to? How much is visual and how much is spoken? What is the trade-off between personal devices and devices that are built into the infrastructure?
• Local wireless network. Today there are wireless networks with different design points for the home, office, mall, and airport. If we had a uniform assumption about the desirable network infrastructure: a high bandwidth fiber backbone and ample wireless networks to reach into that backbone, what would the wireless piece look like? What would it cost to configure? Do we have the right protocols designed for that set of assumptions? Spectrum?
• Spectrum. How should spectrum be allocated to balance all of the wireless needs between local area, metropolitan, and wide area?
• Wide area backbone network design. Similar to the wireless network observation. We have fiber networks today. Let’s create a specification to understand the needs of tomorrow. What would represent sufficient bandwidth for foreseeable needs? How would we build such a network? Cost?
While core network design is principally about systems engineering - improvement in core communications research technology will also improve the core network. Here are some of these core research areas motivated by the desire to create a core infrastructure for secure natural communications.
Sample core infrastructure research issues
• Fiber componentry. Getting more bits per second through a fiber link.
• Fiber network architecture. Designing the layout of fiber multiplexors and processors to improve network bandwidth
• Nanotechnology. Continued miniaturization is critical for sensor technology
• Wireless contention algorithms. Better signal processing to get more bandwidth out of limited spectrum
• Multi-media. How to utilize network bandwidth to address quality of service needs for disparate traffic patterns such as voice, data, and video.
Wednesday, March 17, 2010
The decode engine
Continuing in the inventory of research problems in secure natural communications.
The player experience has two components. Players physically send and receive bits of information and the semantics are understood naturally. The decode engine is the proxy for the player that allows the information infrastructure to interpret the user desire and allow for natural behavior.
The level of sophistication in the decode engine depends on the depth and breadth of capability. In the most basic level, we use today’s keyboard and display and coded commands and the decode engine barely exists. But this is not “natural”. So the decode engine must become sophisticated to enable the natural behavior at both the semantic and physical layers.
Research problems related to the decode engine
• Core speech recognition algorithms. The most natural way that we communicate is by speaking. Speech recognition has made enormous progress but it is a very hard problem to get 100% so there will always be room for improvement.
• Networked speech recognition. There are efficiencies to performing some speech recognition locally, at the device that is picking up the sound itself. This especially is the case if the microphone is in a room attached to a processing chip. However, if the microphone is tiny – a wearable – then there must be a way to find processing and stream the bits to the processor.
• Handwriting recognition. Less generally used that speech recognition but a similar application and problem.
• Gesture recognition. For some, gestures enhance the content of the message. For others, such as speech impaired, the gesture language is the language.
• Natural language recognition. Another hard problem! A natural way to communicate is to express in a sentence – rather than a structured form – what function one wants to achieve.
• Language translation. The Internet and World Wide Web are global. We need translation capability that provides the full power of the web to speakers of any language.
• Searching unstructured data. We now know that search is a fundamental piece of the infrastructure.
• Query by image content. There are images on the web. These should be searchable.
• Query by video content. There is video on the web. This should be searchable.
• Separating commands from data. As we communicate naturally we provide bits – either in text form, or speech, gesture, etc. Some of these bits are the content and some are the command structure. For text, we know how to separate the two. For more natural forms such as speech, we need to find approaches to make this separation as natural as possible.
The player experience has two components. Players physically send and receive bits of information and the semantics are understood naturally. The decode engine is the proxy for the player that allows the information infrastructure to interpret the user desire and allow for natural behavior.
The level of sophistication in the decode engine depends on the depth and breadth of capability. In the most basic level, we use today’s keyboard and display and coded commands and the decode engine barely exists. But this is not “natural”. So the decode engine must become sophisticated to enable the natural behavior at both the semantic and physical layers.
Research problems related to the decode engine
• Core speech recognition algorithms. The most natural way that we communicate is by speaking. Speech recognition has made enormous progress but it is a very hard problem to get 100% so there will always be room for improvement.
• Networked speech recognition. There are efficiencies to performing some speech recognition locally, at the device that is picking up the sound itself. This especially is the case if the microphone is in a room attached to a processing chip. However, if the microphone is tiny – a wearable – then there must be a way to find processing and stream the bits to the processor.
• Handwriting recognition. Less generally used that speech recognition but a similar application and problem.
• Gesture recognition. For some, gestures enhance the content of the message. For others, such as speech impaired, the gesture language is the language.
• Natural language recognition. Another hard problem! A natural way to communicate is to express in a sentence – rather than a structured form – what function one wants to achieve.
• Language translation. The Internet and World Wide Web are global. We need translation capability that provides the full power of the web to speakers of any language.
• Searching unstructured data. We now know that search is a fundamental piece of the infrastructure.
• Query by image content. There are images on the web. These should be searchable.
• Query by video content. There is video on the web. This should be searchable.
• Separating commands from data. As we communicate naturally we provide bits – either in text form, or speech, gesture, etc. Some of these bits are the content and some are the command structure. For text, we know how to separate the two. For more natural forms such as speech, we need to find approaches to make this separation as natural as possible.
Wednesday, March 10, 2010
The player's physical experience
I apologize that this week’s posting was a bit delayed. I became somewhat busy with a new position. Check out http://www.w3.org/2010/03/ceo-pr.html for details.
We have been wandering through the 5-6 key constructs for secure natural communications. One of my key objectives is to enumerate research areas that require more effort to achieve our goal. Comments are invited on current status and research progress.
If the player’s semantic experience is one of higher-level semantics of unlimited breadth, the physical experience deals with a more specific set of modes of physical interaction that a user might have with the infrastructure. For each mode, the challenge is the same. The player has some means of physically expressing themselves. What is most “natural” depends on the player and depends on the application. But in any case, there are only a limited number of methods that are used.
Research problems related to the player’s physical experience
• Sensors. To obtain a greater degree of naturalness there will be many core sensors in the infrastructure. These will range from traffic sensors on highways to cameras to speech sensitive devices. Research is required to improve the cost, shrink the size, and blend into the environment.
• Enabling everything for secure natural communications. Further reductions in costs for RFID’s and GPS will include more items in the intelligent network and have their location trackable.
• Speech sensors. An important set of sensors. Microphones are available as part of the infrastructure so that people’s speech can be the primary input device of their intentions. Improved noise and echo cancellation for the overall system is required.
• Wearables. A key design point for small sensors; particularly speech capture is a wearable sensor. This requires research in nanotechnology to better embed this capability into garments.
• Video, gesture capture. This will enable a richer interpretation of the user’s intent, and a richer understanding of what is going on at target locations.
• Other methods. These would include different forms of keyboard, signing (for speech impaired), etc.
We have been wandering through the 5-6 key constructs for secure natural communications. One of my key objectives is to enumerate research areas that require more effort to achieve our goal. Comments are invited on current status and research progress.
If the player’s semantic experience is one of higher-level semantics of unlimited breadth, the physical experience deals with a more specific set of modes of physical interaction that a user might have with the infrastructure. For each mode, the challenge is the same. The player has some means of physically expressing themselves. What is most “natural” depends on the player and depends on the application. But in any case, there are only a limited number of methods that are used.
Research problems related to the player’s physical experience
• Sensors. To obtain a greater degree of naturalness there will be many core sensors in the infrastructure. These will range from traffic sensors on highways to cameras to speech sensitive devices. Research is required to improve the cost, shrink the size, and blend into the environment.
• Enabling everything for secure natural communications. Further reductions in costs for RFID’s and GPS will include more items in the intelligent network and have their location trackable.
• Speech sensors. An important set of sensors. Microphones are available as part of the infrastructure so that people’s speech can be the primary input device of their intentions. Improved noise and echo cancellation for the overall system is required.
• Wearables. A key design point for small sensors; particularly speech capture is a wearable sensor. This requires research in nanotechnology to better embed this capability into garments.
• Video, gesture capture. This will enable a richer interpretation of the user’s intent, and a richer understanding of what is going on at target locations.
• Other methods. These would include different forms of keyboard, signing (for speech impaired), etc.
Tuesday, March 2, 2010
Inventory of areas related to the player's semantic experience
Review of purpose of this blog
I’ve described the concept of Secure Natural Communications, why it is broadly achievable, and the benefits. I’ve described 5-6 key constructs: the player’s semantic experience, the player’s physical experience, the decode engine, core infrastructure, information intelligence, and advanced applications.
To move forward requires an inventory of the technical areas that will require more development to perfect secure natural communications. We rely on numerous technologies – all of them are good enough to get started; most of them would benefit from further enhancement. So after we inventory these areas, we will need to assess our current level of capability in these areas and then create a roadmap to improve capability.
Today’s blog begins the inventory.
Technologies related to a player’s semantic experience
Today’s technology already has many of the components needed to provide a natural semantic interface for the player. Each will be enhanced further and customized to provide secure natural communications. Here are some of the areas that require attention.
• Networked consumer devices. There already has been tremendous progress in making consumer devices for music, video, telephony, and computing to be more universal (capability) and networked. Increasingly, their user interfaces will adapt to the interfaces that people find most natural. This progress will continue. More focus will be necessary on standardization and integration.
• Multiple communication threads. Inherent in the notion of group communications is that a person will be involved in multiple communications threads simultaneously. For the player’s semantic experience the infrastructure must be able to surface these multiple threads to people irrespective of how they interact with the infrastructure. We have some technologies to do this for a person sitting at their display. More challenging will be if they are using shared video screens in the infrastructure, or if they are trying to communicate naturally without devices.
• Higher-level semantics. This is too large an area to deal with comprehensively in this paper, but ultimately it is the broadest part of secure natural communications. This paper limits its focus to infrastructure common to all applications – not the semantics of each application. However, each application area requires its own standardization on semantics. Example areas are:
o Non-communications infrastructure intensive. Examples include numerical applications, devtest, ERP, and database.
o Base-level communications infrastructure. Examples include Web, file, and transaction processing
o Applications that require their own infrastructure. This is the most interesting area for higher-level semantics. There are some applications that are so communications intensive that they require their own sophisticated semantic infrastructure to be provided by an overall infrastructure. These would include social networking, collaboration, and virtual desktop.
I’ve described the concept of Secure Natural Communications, why it is broadly achievable, and the benefits. I’ve described 5-6 key constructs: the player’s semantic experience, the player’s physical experience, the decode engine, core infrastructure, information intelligence, and advanced applications.
To move forward requires an inventory of the technical areas that will require more development to perfect secure natural communications. We rely on numerous technologies – all of them are good enough to get started; most of them would benefit from further enhancement. So after we inventory these areas, we will need to assess our current level of capability in these areas and then create a roadmap to improve capability.
Today’s blog begins the inventory.
Technologies related to a player’s semantic experience
Today’s technology already has many of the components needed to provide a natural semantic interface for the player. Each will be enhanced further and customized to provide secure natural communications. Here are some of the areas that require attention.
• Networked consumer devices. There already has been tremendous progress in making consumer devices for music, video, telephony, and computing to be more universal (capability) and networked. Increasingly, their user interfaces will adapt to the interfaces that people find most natural. This progress will continue. More focus will be necessary on standardization and integration.
• Multiple communication threads. Inherent in the notion of group communications is that a person will be involved in multiple communications threads simultaneously. For the player’s semantic experience the infrastructure must be able to surface these multiple threads to people irrespective of how they interact with the infrastructure. We have some technologies to do this for a person sitting at their display. More challenging will be if they are using shared video screens in the infrastructure, or if they are trying to communicate naturally without devices.
• Higher-level semantics. This is too large an area to deal with comprehensively in this paper, but ultimately it is the broadest part of secure natural communications. This paper limits its focus to infrastructure common to all applications – not the semantics of each application. However, each application area requires its own standardization on semantics. Example areas are:
o Non-communications infrastructure intensive. Examples include numerical applications, devtest, ERP, and database.
o Base-level communications infrastructure. Examples include Web, file, and transaction processing
o Applications that require their own infrastructure. This is the most interesting area for higher-level semantics. There are some applications that are so communications intensive that they require their own sophisticated semantic infrastructure to be provided by an overall infrastructure. These would include social networking, collaboration, and virtual desktop.
Wednesday, February 24, 2010
The player's semantic experience
In my first posting I outlined what it means for a player to communicate at will without distance or devices. Just as we have conversations with people in the room we are in and our friends hear our talking and respond, that in the future we can have the same natural interaction when we want to accomplish anything digital. The excitement of this statement is when we interpret “anything” as a “rich communications experience”. We are no longer restricted to verbal communications between two people. Secure natural communications includes all communications and computing interactions that exist for people and devices - for human communications as well as device communications.
I also introduced the five key constructs required to achieve this new capability and infrastructure. We require support for the player’s semantic experience, the player’s physical experience, the decode engine, core infrastructure, and information intelligence. If you have been following some of the dialog on this blog you know that Chiku has proposed that we should also focus on new enabled applications as a sixth construct.
In the next set of postings I will elaborate on each of these constructs and also discuss some of the research problems that can be addressed to further improve capabilities
The player’s semantic experience
To better characterize how the player will intellectually interact with other players, we characterize the following components of the player’s semantic experience. In the infrastructure we build for secure natural communications we will require support for each of these components.
• The players. The players in secure natural communications are the potential parties to communications. In today’s terms it would be anything with an IP address. This includes people, computers, devices, sensors, microphones, cameras, items with RFID tags, MP3 players, anything with a GPS system attached, scanners, medical imaging devices, etc. So the notion of “natural” communications means that each of these devices is able to send and receive communications in a way that is natural for them. And any user is able to naturally refer to these items and find them.
• The groups. Group communication has become a fundamental paradigm through the advent of social networking. The infrastructure must make it natural for users to participate in arbitrary ad hoc groups.
• Media. Secure natural communications must support unstructured data as well as voice, video, high quality music, fax, etc.
• Semantics. We have developed increasingly sophisticated means for people, computer systems, and applications to communicate with each other in a highly coded fashion. To the extent that the current protocols and interfaces are adequate, they would be used as they are defined today; but many need to be made more natural.
With these four components, we can give the next level of description of secure natural communications.
Since we include all players imaginable this clarifies the form of communications that is supported. Any player can communicate with any other player. Each has a set of requests supported and each has a different sense of what is “natural”. The infrastructure supports both their physical requests and semantic meeting thereof.
“Groups” recognizes that much communication wants to be multi-party. Increasingly, large projects cross organizational boundaries and collaborate on some set of tasks. Hence all types of group requests must be intelligently anticipated by the infrastructure. When a person or device is participating in multiple group interactions, they must be able to handle the challenge to participate in multiple groups. To achieve this, the infrastructure must present all of the information from multiple interactions in the most intuitive possible fashion. They must handle multiple group interactions digitally as well as any human can handle multiple physical interactions. The infrastructure needs to support an exponential number of potential groups that want to form.
“Media” requires that we digitize everything, so we are encompassing all of human experience in this infrastructure.
The Semantic layer implies that compute tasks that happen with a regular pattern each develop their own means of semantic expression which is also supported.
Of course, in describing each of these components of the player’s semantic experience, I am only scratching the surface. As we develop this, each of these will get more detail. The readers of the blog are invited to provide their viewpoints by commenting on what they think some of these details should be.
In the next posting, I will describe efforts that should take place to more fully develop a player’s semantic experience.
I also introduced the five key constructs required to achieve this new capability and infrastructure. We require support for the player’s semantic experience, the player’s physical experience, the decode engine, core infrastructure, and information intelligence. If you have been following some of the dialog on this blog you know that Chiku has proposed that we should also focus on new enabled applications as a sixth construct.
In the next set of postings I will elaborate on each of these constructs and also discuss some of the research problems that can be addressed to further improve capabilities
The player’s semantic experience
To better characterize how the player will intellectually interact with other players, we characterize the following components of the player’s semantic experience. In the infrastructure we build for secure natural communications we will require support for each of these components.
• The players. The players in secure natural communications are the potential parties to communications. In today’s terms it would be anything with an IP address. This includes people, computers, devices, sensors, microphones, cameras, items with RFID tags, MP3 players, anything with a GPS system attached, scanners, medical imaging devices, etc. So the notion of “natural” communications means that each of these devices is able to send and receive communications in a way that is natural for them. And any user is able to naturally refer to these items and find them.
• The groups. Group communication has become a fundamental paradigm through the advent of social networking. The infrastructure must make it natural for users to participate in arbitrary ad hoc groups.
• Media. Secure natural communications must support unstructured data as well as voice, video, high quality music, fax, etc.
• Semantics. We have developed increasingly sophisticated means for people, computer systems, and applications to communicate with each other in a highly coded fashion. To the extent that the current protocols and interfaces are adequate, they would be used as they are defined today; but many need to be made more natural.
With these four components, we can give the next level of description of secure natural communications.
Since we include all players imaginable this clarifies the form of communications that is supported. Any player can communicate with any other player. Each has a set of requests supported and each has a different sense of what is “natural”. The infrastructure supports both their physical requests and semantic meeting thereof.
“Groups” recognizes that much communication wants to be multi-party. Increasingly, large projects cross organizational boundaries and collaborate on some set of tasks. Hence all types of group requests must be intelligently anticipated by the infrastructure. When a person or device is participating in multiple group interactions, they must be able to handle the challenge to participate in multiple groups. To achieve this, the infrastructure must present all of the information from multiple interactions in the most intuitive possible fashion. They must handle multiple group interactions digitally as well as any human can handle multiple physical interactions. The infrastructure needs to support an exponential number of potential groups that want to form.
“Media” requires that we digitize everything, so we are encompassing all of human experience in this infrastructure.
The Semantic layer implies that compute tasks that happen with a regular pattern each develop their own means of semantic expression which is also supported.
Of course, in describing each of these components of the player’s semantic experience, I am only scratching the surface. As we develop this, each of these will get more detail. The readers of the blog are invited to provide their viewpoints by commenting on what they think some of these details should be.
In the next posting, I will describe efforts that should take place to more fully develop a player’s semantic experience.
Tuesday, February 16, 2010
Secure natural communications - achievability and relationship with computing
Today I’ll address two questions that continue to come up.
First: with secure natural communications people can walk around the streets not holding any computing device and look up some piece of data or request a computation from the computing cloud, and it gets performed by an intelligent infrastructure. I’m asked - how realistic is this? People struggle to understand the vastness of the infrastructure needs to achieve this lofty goal. Are parts of this “vision” beyond our grasp?
Here is how I think about it. There are more and less demanding variations on secure natural communications. Enough technology exists that a proof-of-concept could be built, today. Over a long period of time technology and infrastructure advances will make more of this possible. Key to making this happen, however, is to set a stretch goal so we can plan the required physical infrastructure and motivate the necessary innovation.
Second question: Information infrastructure, as in today’s wireless infrastructure, Internet or World Wide Web provides more than communications. Most of the value is in the applications - e-commerce, auctions, advertising, document interchange, compute intensive applications, storage. How does our focus on communications relate to the larger computing applications.
Here is how I think about this issue. At the moment, this blog is focusing on the core infrastructure rather than the compute tasks because for each of these tasks, there is a dominant common communications infrastructure need. To be sure, communications takes place at every level: from physical transmission of bits at the lowest level to the interpretation thereof by applications that have application level protocols.
In particular, when we discuss “the player’s semantic experience”, we will ultimately support the semantics for every type of computing task. However, with the existence of millions of applications this blog will not address application specific protocols herein – although some require substantial infrastructure of their own. That work must proceed in parallel with the core communications infrastructure.
Next blog posting I will next move to a detailed description of the five constructs mentioned in the introductory posting.
First: with secure natural communications people can walk around the streets not holding any computing device and look up some piece of data or request a computation from the computing cloud, and it gets performed by an intelligent infrastructure. I’m asked - how realistic is this? People struggle to understand the vastness of the infrastructure needs to achieve this lofty goal. Are parts of this “vision” beyond our grasp?
Here is how I think about it. There are more and less demanding variations on secure natural communications. Enough technology exists that a proof-of-concept could be built, today. Over a long period of time technology and infrastructure advances will make more of this possible. Key to making this happen, however, is to set a stretch goal so we can plan the required physical infrastructure and motivate the necessary innovation.
Second question: Information infrastructure, as in today’s wireless infrastructure, Internet or World Wide Web provides more than communications. Most of the value is in the applications - e-commerce, auctions, advertising, document interchange, compute intensive applications, storage. How does our focus on communications relate to the larger computing applications.
Here is how I think about this issue. At the moment, this blog is focusing on the core infrastructure rather than the compute tasks because for each of these tasks, there is a dominant common communications infrastructure need. To be sure, communications takes place at every level: from physical transmission of bits at the lowest level to the interpretation thereof by applications that have application level protocols.
In particular, when we discuss “the player’s semantic experience”, we will ultimately support the semantics for every type of computing task. However, with the existence of millions of applications this blog will not address application specific protocols herein – although some require substantial infrastructure of their own. That work must proceed in parallel with the core communications infrastructure.
Next blog posting I will next move to a detailed description of the five constructs mentioned in the introductory posting.
Thursday, February 11, 2010
Secure natural commmunications - elaboration
Until now, my description of secure natural communications has been conceptual, with a few examples. Before we go further it is useful to tease out more of what the words mean in the original statement describing secure natural communications:
Any player (person or device) can have a rich communications experience at will with any target without encumbrances of distance or additional devices.
Here are some elaborations of the above goal.
A rich communications experience. Included in this would be:
• Finding someone I’ve never communicated with
• Having a record of the relevant threads of information that I previously have with this group.
• Having the universe of social networking capabilities at my fingertips.
• In particular, sharing all types of data, using all types of media
• Being able to tie in the history of communications that are relevant.
• A range: from purely communicating to requesting a complex computing task with sophisticated semantics
• Security and privacy
…at will. Included in this would be:
• I don’t need to plan to find addressing conventions to locate the other player.
• I can use my natural means of communicating and a sensory rich world will understand my physical gestures and understand the semantics
• Speech recognition, gestures, natural language, are supported
• All media types are supported
• There are means (such as pervasive video screens or wearables) for me to get responses
…with any target. Included in this are groups, so it would include:
• Being able to create new groups with security and privacy
• Being able to create ad hoc groups to achieve some new task without a great deal of overhead – and still have the rich communications experience.
• Being able to join groups and get access to their shared history
…without encumbrances of distance. Included in this is:
• Assuming the very high bandwidth backbone in which bandwidth is not a constraint.
• Assuming the very high bandwidth local wireless network that gives adequate untethered bandwidth to get to the high bandwidth fiber backbone
• Developing protocols that work well with this unlimited bandwidth
• Recognizing that for certain purposes there are still constraints of propagation delay
…or additional devices• Implies that there are natural user interfaces (speech, gesture) that can be relied on; i.e. the technology is good enough
• Implies that when using such natural interfaces in a noisy area that my specific sentences and commands will be heard
• Understands that this might require nano-scale wearable microphones
• Any language
• Recognizes that many people will want to continue to use keyboard for input
• Recognizes that a variety of display and storage devices will continue to be needed
This provides a more complete description by example of the type of capability that is desired. Next posting I’ll provide a little more general background, before moving to a more detailed discussion of the constructs that can deliver this and the technologies that are needed.
Any player (person or device) can have a rich communications experience at will with any target without encumbrances of distance or additional devices.
Here are some elaborations of the above goal.
A rich communications experience. Included in this would be:
• Finding someone I’ve never communicated with
• Having a record of the relevant threads of information that I previously have with this group.
• Having the universe of social networking capabilities at my fingertips.
• In particular, sharing all types of data, using all types of media
• Being able to tie in the history of communications that are relevant.
• A range: from purely communicating to requesting a complex computing task with sophisticated semantics
• Security and privacy
…at will. Included in this would be:
• I don’t need to plan to find addressing conventions to locate the other player.
• I can use my natural means of communicating and a sensory rich world will understand my physical gestures and understand the semantics
• Speech recognition, gestures, natural language, are supported
• All media types are supported
• There are means (such as pervasive video screens or wearables) for me to get responses
…with any target. Included in this are groups, so it would include:
• Being able to create new groups with security and privacy
• Being able to create ad hoc groups to achieve some new task without a great deal of overhead – and still have the rich communications experience.
• Being able to join groups and get access to their shared history
…without encumbrances of distance. Included in this is:
• Assuming the very high bandwidth backbone in which bandwidth is not a constraint.
• Assuming the very high bandwidth local wireless network that gives adequate untethered bandwidth to get to the high bandwidth fiber backbone
• Developing protocols that work well with this unlimited bandwidth
• Recognizing that for certain purposes there are still constraints of propagation delay
…or additional devices• Implies that there are natural user interfaces (speech, gesture) that can be relied on; i.e. the technology is good enough
• Implies that when using such natural interfaces in a noisy area that my specific sentences and commands will be heard
• Understands that this might require nano-scale wearable microphones
• Any language
• Recognizes that many people will want to continue to use keyboard for input
• Recognizes that a variety of display and storage devices will continue to be needed
This provides a more complete description by example of the type of capability that is desired. Next posting I’ll provide a little more general background, before moving to a more detailed discussion of the constructs that can deliver this and the technologies that are needed.
Thursday, February 4, 2010
Secure natural communications - how to proceed
After my initial posting last week, the most frequent question that I received was: how do I plan to proceed with this project? Is this something that we will build tomorrow? Is there some spec? Are these some wandering ideas?
Most of these questions came through email from my colleagues. This blog should be an active discussion community so please post your questions and comments to the blog – as several people have already done.
Reprise – and what is possible today
To reprise, the world should provide an infrastructure where: people can have a rich communications experience at will.
From a core technology perspective, almost all technologies required exist today in elemental form. There are numerous sensors that exist: often called by names such as microphones, cameras, displays, etc. These sensors can detect speech, movement, typing or provide images and videos. Properly configured, they provide the elements of natural human communications overcoming individual device conventions. With ample communications bandwidth we also overcome distance constraints. An infrastructure could both sense bits and interpret them – although not with perfection.
From this - in principle – one can take people’s utterances and build a management system that acts upon them. These can be primitive utterances that request basic communications. Or coded machine commands that request the performance of sophisticated compute function, rich applications, moving function to the cloud, etc.
What is not possible today
There is no infrastructure that provides secure natural communications. Not across the globe; not in a country; not in a city – not even on a campus. It would be considerably expensive to build and there is no known economic model to pay for this new infrastructure.
Goal and method
Here is my pitch. If it is possible to create such an infrastructure – and it would mimic how people naturally communicate – isn’t that compelling enough?
Hence my quest. I want to use a series of postings to express my views about the components of such an infrastructure. I need experts to add precision to this with their knowledge and views. After a while, we will develop enough clarity of viewpoint that we can prototype it on a small campus. If people like it – we will grow it from there.
Next several postings
So my first few postings will continue to discuss the high level concept of secure natural communications. I want to engage in discussion on the general idea. After that, I will elaborate in detail on the five constructs that I outlined last week. Provide a more complete description. What technologies exist? What more do we need to develop over time? And in discussion I will continue to need experts who have the domain knowledge to add precision.
Most of these questions came through email from my colleagues. This blog should be an active discussion community so please post your questions and comments to the blog – as several people have already done.
Reprise – and what is possible today
To reprise, the world should provide an infrastructure where: people can have a rich communications experience at will.
From a core technology perspective, almost all technologies required exist today in elemental form. There are numerous sensors that exist: often called by names such as microphones, cameras, displays, etc. These sensors can detect speech, movement, typing or provide images and videos. Properly configured, they provide the elements of natural human communications overcoming individual device conventions. With ample communications bandwidth we also overcome distance constraints. An infrastructure could both sense bits and interpret them – although not with perfection.
From this - in principle – one can take people’s utterances and build a management system that acts upon them. These can be primitive utterances that request basic communications. Or coded machine commands that request the performance of sophisticated compute function, rich applications, moving function to the cloud, etc.
What is not possible today
There is no infrastructure that provides secure natural communications. Not across the globe; not in a country; not in a city – not even on a campus. It would be considerably expensive to build and there is no known economic model to pay for this new infrastructure.
Goal and method
Here is my pitch. If it is possible to create such an infrastructure – and it would mimic how people naturally communicate – isn’t that compelling enough?
Hence my quest. I want to use a series of postings to express my views about the components of such an infrastructure. I need experts to add precision to this with their knowledge and views. After a while, we will develop enough clarity of viewpoint that we can prototype it on a small campus. If people like it – we will grow it from there.
Next several postings
So my first few postings will continue to discuss the high level concept of secure natural communications. I want to engage in discussion on the general idea. After that, I will elaborate in detail on the five constructs that I outlined last week. Provide a more complete description. What technologies exist? What more do we need to develop over time? And in discussion I will continue to need experts who have the domain knowledge to add precision.
Friday, January 29, 2010
Secure Natural Communications – Initial posting
Sitting in a lecture about global warming you have a new insight about the climate model you have been quietly building in your office, twelve time zones away. You mutter under your breath; let me change these variables, re-run my model, and see if my new results change how we deal with this pressing issue. As soon as the words come out of your mouth, the three dimensional plot of the new result appears on the personal display in the conference center. One example of secure natural communications.
You run into a college roommate on line. You are reminiscing about a third roommate. You say: I would like to talk to him. Although you don’t know where they are, your mere request to speak to this person causes him to be found and for the conversation to start.
Secure natural communications
Over the past twenty years, society has benefited from and been transformed by new infrastructures. Wireless infrastructure and cell phones have made us instantly accessible. The Internet and the World-Wide Web have changed the way we obtain information, interact with others, and shop. The next transformation will come from a far more intelligent infrastructure. Individuals will have a rich communications experience at will. People will be liberated from managing devices, constraints of distance and location, and knowledge of arcane commands. With foresight, we will build security into this infrastructure so privacy is restored.
We call this future vision “Secure natural communications”. It begins with a simple premise which provides a powerful result. The simple expression is:
Any player (person or device) can have a rich communications experience at will with any target without encumbrances of distance or additional devices.
We use the word player to refer to any person or device that needs to communicate with any other set of players.
People have many input / output capabilities – including speech, gesture, feel, vision, keyboard, etc. Devices also have their built-in input / output capabilities. Whenever a player wants to accomplish something which involves another set of players which has an information component, they can do so. An information component could be any communications or computing task imaginable.
Five key constructs: First, the player’s semantic experience is natural. That means, that they communicate – or request that a task be performed in their most intuitive fashion. They use exactly the same words, sentences, and nuances they would use if they were analogously trying to accomplish something in the physical world. The meaning of their action in the digital world matches what it would mean in the physical world.
The physical world is fully enabled and prepared to do what it must at the physical level to provide this semantic experience. Thus the player has a natural physical experience as well. Not only do they express the task with the words and sentences that are natural, but they use their preferred personal input / output modality: whether speech, keyboard, handwriting, gesture, video screen, etc. Sensors are ubiquitous to detect and display what is going on so that the expressive piece of the requester is natural and receptor piece of the player(s) being communicated with are equally natural.
The more natural the player’s experience at the semantic and physical level, the more intelligence required by the infrastructure to figure out what the player wants to accomplish. That is what we call the decode engine.
Aside from a decode engine that translates intent into action, there must be sufficient capacity to achieve everyone’s wishes. Core infrastructure is the design and specification of the aggregate capacity needed.
With this adequate capacity there needs to be a management operations system that allocates capacity to the requested tasks in an efficient manner. This is what we call information intelligence.
Purpose of this blog
I’ve been thinking about these ideas for a long time and part of my time will be to explore them further. Others have similar ideas and we need a place to come together to develop them. So this blogspot becomes a place where a community of people across different organizations come together to share ideas about future infrastructure.
I have ideas to get us started. The roughest outline is posted above. In my next several postings I will develop them further. But at each step I invite people to comment so we can make this everyone’s idea of the future.
To get from where we are today to a vision of the future requires a great deal. First it requires a well socialized vision that we can all agree upon. But that vision will rely on technology that does not yet exist. So part of the objective is to create a research agenda spanning several industries: IT, telecommunications, and nanotechnology with a very focused end goal.
At some point, the blog format will become unwieldy. At that point we’ll move to a wiki or some other sharing mechanism.
As we solve technical problems to create
this exciting future we will need to build prototypes and ultimately build out the infrastructure. That will be another topic we work our way through.
Where did this come from?
A bit about my background. I’ve been fortunate in my career to get the broadest exposure possible to a palate of technologies. At IBM Research for over 16 years, we explored the limits of the IT industry, and as VP of Systems and Software I learned, participated in, and explored the entire range of computer system and software technology. Later, as IBM Corporate VP of Technology, I assessed IBM’s entire technology base – including semiconductors, storage, systems, software, and services.
Later, as President of Research at Bell Laboratories, I had a similar experience and exposure – here with a stronger emphasis on telecommunications, nanotechnology, and physical sciences. Towards the end of my tenure at Bell, I gave a few papers on secure natural communications. It was the coming together of all the advances in two industries – projected a few years in the future – and bolstered by technology making new things possible.
I didn’t pursue it further. Many of the technologies were too early. The vision was too broad. How could one construct an effort to bring all of these technologies together.
I took a position as Executive Vice President and CTO of Novell. Novell, a leading Linux distributor participates heavily in the open source community. I saw the power. I saw how communities of like minded people can build operating systems, database systems, app servers, virtualization solutions, etc.
Here I am today, armed with this vision that unifies IT, telco, and core technology by creating a new infrastructure for our future. I see how this can be accomplished with the power of communities. I resigned my position at Novell. One of several objectives I now have is to now to create a community to drive secure natural communications. We start today.
Call for action
What am I requesting from you? First, participate. Give your viewpoints on the vision. Add your own vision. My starting blogs necessarily will evolve as great minds add their ideas
Second, invite your colleagues to participate. We want as many viewpoints as possible.
Third, have some respect for different viewpoints. The blogosphere can be unvarnished. I think we are all better off when we listen to others. There is a reason that we all have two ears and only one mouth.
You run into a college roommate on line. You are reminiscing about a third roommate. You say: I would like to talk to him. Although you don’t know where they are, your mere request to speak to this person causes him to be found and for the conversation to start.
Secure natural communications
Over the past twenty years, society has benefited from and been transformed by new infrastructures. Wireless infrastructure and cell phones have made us instantly accessible. The Internet and the World-Wide Web have changed the way we obtain information, interact with others, and shop. The next transformation will come from a far more intelligent infrastructure. Individuals will have a rich communications experience at will. People will be liberated from managing devices, constraints of distance and location, and knowledge of arcane commands. With foresight, we will build security into this infrastructure so privacy is restored.
We call this future vision “Secure natural communications”. It begins with a simple premise which provides a powerful result. The simple expression is:
Any player (person or device) can have a rich communications experience at will with any target without encumbrances of distance or additional devices.
We use the word player to refer to any person or device that needs to communicate with any other set of players.
People have many input / output capabilities – including speech, gesture, feel, vision, keyboard, etc. Devices also have their built-in input / output capabilities. Whenever a player wants to accomplish something which involves another set of players which has an information component, they can do so. An information component could be any communications or computing task imaginable.
Five key constructs: First, the player’s semantic experience is natural. That means, that they communicate – or request that a task be performed in their most intuitive fashion. They use exactly the same words, sentences, and nuances they would use if they were analogously trying to accomplish something in the physical world. The meaning of their action in the digital world matches what it would mean in the physical world.
The physical world is fully enabled and prepared to do what it must at the physical level to provide this semantic experience. Thus the player has a natural physical experience as well. Not only do they express the task with the words and sentences that are natural, but they use their preferred personal input / output modality: whether speech, keyboard, handwriting, gesture, video screen, etc. Sensors are ubiquitous to detect and display what is going on so that the expressive piece of the requester is natural and receptor piece of the player(s) being communicated with are equally natural.
The more natural the player’s experience at the semantic and physical level, the more intelligence required by the infrastructure to figure out what the player wants to accomplish. That is what we call the decode engine.
Aside from a decode engine that translates intent into action, there must be sufficient capacity to achieve everyone’s wishes. Core infrastructure is the design and specification of the aggregate capacity needed.
With this adequate capacity there needs to be a management operations system that allocates capacity to the requested tasks in an efficient manner. This is what we call information intelligence.
Purpose of this blog
I’ve been thinking about these ideas for a long time and part of my time will be to explore them further. Others have similar ideas and we need a place to come together to develop them. So this blogspot becomes a place where a community of people across different organizations come together to share ideas about future infrastructure.
I have ideas to get us started. The roughest outline is posted above. In my next several postings I will develop them further. But at each step I invite people to comment so we can make this everyone’s idea of the future.
To get from where we are today to a vision of the future requires a great deal. First it requires a well socialized vision that we can all agree upon. But that vision will rely on technology that does not yet exist. So part of the objective is to create a research agenda spanning several industries: IT, telecommunications, and nanotechnology with a very focused end goal.
At some point, the blog format will become unwieldy. At that point we’ll move to a wiki or some other sharing mechanism.
As we solve technical problems to create
this exciting future we will need to build prototypes and ultimately build out the infrastructure. That will be another topic we work our way through.
Where did this come from?
A bit about my background. I’ve been fortunate in my career to get the broadest exposure possible to a palate of technologies. At IBM Research for over 16 years, we explored the limits of the IT industry, and as VP of Systems and Software I learned, participated in, and explored the entire range of computer system and software technology. Later, as IBM Corporate VP of Technology, I assessed IBM’s entire technology base – including semiconductors, storage, systems, software, and services.
Later, as President of Research at Bell Laboratories, I had a similar experience and exposure – here with a stronger emphasis on telecommunications, nanotechnology, and physical sciences. Towards the end of my tenure at Bell, I gave a few papers on secure natural communications. It was the coming together of all the advances in two industries – projected a few years in the future – and bolstered by technology making new things possible.
I didn’t pursue it further. Many of the technologies were too early. The vision was too broad. How could one construct an effort to bring all of these technologies together.
I took a position as Executive Vice President and CTO of Novell. Novell, a leading Linux distributor participates heavily in the open source community. I saw the power. I saw how communities of like minded people can build operating systems, database systems, app servers, virtualization solutions, etc.
Here I am today, armed with this vision that unifies IT, telco, and core technology by creating a new infrastructure for our future. I see how this can be accomplished with the power of communities. I resigned my position at Novell. One of several objectives I now have is to now to create a community to drive secure natural communications. We start today.
Call for action
What am I requesting from you? First, participate. Give your viewpoints on the vision. Add your own vision. My starting blogs necessarily will evolve as great minds add their ideas
Second, invite your colleagues to participate. We want as many viewpoints as possible.
Third, have some respect for different viewpoints. The blogosphere can be unvarnished. I think we are all better off when we listen to others. There is a reason that we all have two ears and only one mouth.
Subscribe to:
Posts (Atom)