在今年3月份RWP中国之旅技术分享活动期间Andrew大师和AWR之父Graham接受了记者采访,首次对外讲述自己的职业生涯和心路历程,分享了他们在中国的感受和期待,并就当前IT系统现状做了深入分析,表达了RWP团队的使命和宗旨,同时也对广大技术爱好和追求者给出了中肯的建议和指导。
本文为采访原文,若需要阅读中文版,请查看:
Xue:First of all, it is an honor to have to you here one more time in China, this time in Beijing. People around the world knows about your career, your specialty and more, but perhaps they do not know how you start to be involved in Oracle Technology, can you tell us some about it?
Graham: I got involved because I worked for software company at that time, I was doing a project we needed to use a database, we analyzed the database around the world,we found this secret database named Oracle, it was very new, it was back in 1984,a very long time ago, and at that time Oracle was very new, version 2 just published, so we did this project using the Oracle database to processing using SQL, when they came to do the nest release of the project, I have decided that I would rather to work for Oracle instead of work for the previous company, so I left the software house and I joined ,and started working with the Oracle Database. Using the power of SQL , manipulating data.
Xue:thank you, that was interesting. And how about you, Mr.holdsworth?
Andrew: I joined the Oracle two or three years after Graham, and that two or three years Oracle became a much bigger company, so Ijoined a much bigger company than graham did, Oracle has Grown so much, but I was working in software, in the same way, I was ,you know, help developing applications, and recognized that Oracle was going to be a very important technology in next ten years, so I wasvery happy to be able to join Oracle.
Xue:So may I say for these years, you always know what you are going to do?
Andrew: to be honest, no, I don't even quite know what I am going to do next week.
Graham: yeah, I think for both of us, it's the case, it seems like an interesting technology at the time, and certainly from my point ,oh well I will do these for couple ofyears and see what happens.
Xue: it going pretty well
Graham: yeah, so it is thirty years later, I am still doing this.
Andrew: Oracle is an interesting company, it allow you to challenge yourself, you know, you can work as hard as be ambitious as you want to be.
Xue: yes, it's an absolutely inspiration. And also, speaking of this, I would like to know how do you find the balance between your career and your life, your hobbies?
Andrew: good question, and when I figure it out, I will tell you.
Graham: yeah, I think we are little luckier these days,cause we are not quite as on the frontline with the customers as we used to be, things have to get estimated thesedays before it reaches to Andrew and myself, making it rather easier to get that balance, not frequently having people calling you up with problems.
Andrew: I think it's also as you get older, you get wiser.And when you are in your twenties and thirties , you were rediculous busy, and don't get that much done, as you get older, you need less time to get the same job done, and you get more efficient, so you spend time on the other aspects of the rest of your life.
Graham: yes, one of my ex-bosses, used to say "work smarter not harder", I think it's something both Andrew and I trying do,trying be smarter and better at what we do.
Xue:people gets older, you just get better. Normally you are traveling around the world giving conferences or you stay more in an office, tell us more about it?
Graham: The standard consulting answer is ‘It depends’。
Andrew: yes, it does.
Xue: meaning if you are not called by the customers?
Graham: yes, it tend to be in cycles, like in the last ten days, wehave been in China, and then we are going to Korea. Before this trip I hadn’tbeen out of the office in over month, so it’s varies a lot.
Andrew: Yes, for me, you know I manage the Real World Performancegroup, and it's always nice to be in the office, because when I in the office,I can learn about the new developments taking place in the database and see myteam, I have a global team, so I am trying to see my team in China, and Europeas much as possible, but what I really enjoy is speaking with the customers with their real world performance challenges. Because that's the Real World. When customers explains the challenges they have, to me that's the most enjoyable part of my job. Presentations can be fun, but it's more funto get involved with the customers and their real challenges and see what wecan do to make the performance ten times, hundreds times , thousands timesfaster because we learn so much from our customers with their performance challenges.
So to repeat while I like traveling a lot for work, I like travelingjust meet the customers, it's much more fun than just presenting. Listening,and working to solve problems, that's the fun part of my job. So not always getting on stage, but actually sitting down and working with real data, andmaking things fast, is the fun part of my job.
Xue:So where do you usually stay related to your job? I mean, where are you based?
Andrew:our headquarter is in the California.
Graham: yes, and also about 20miles from San Francisco in California,but we are both origianlly of the England, I am sure you can tell from the accent.
Andrew:we both grow up in England
Graham: and we both started learning Oracle inthe UK, before moving to California.
Xue: We know that traveling has been part of your career and perhaps part of your regular life, do you know inhow many countries you have been in the world ?
Andrew: wow,probably close to 40.
Graham: yes, pretty much around 40, but in last ten years, I am prettysure they are over 30.
Xue : wow, that's amazing to knowmore about your personal life.. great!!
Well,Let’s go with some questions related to your participation in the RWP China tour.According to the Andrew 'sapply for email interview, in any monththere will be RWP training events in every continent taking place. That's everycontinent? Last time I heard is three ,Continent,North America,Europe and Asia?
Andrew: yes, now it's every continent.
Xue: And the training events, youmean more like classes which last forfew days weeks ,or just couple hours?
Andrew: last for couple days.
Xue:In fact,forall the technicians from China,I would like to invite you to tell more about your sharing/training plan?
Andrew: China is very lucky, we have a greatteam in China,so they can teach Real World Performancein Chinese and not have to do deal with the translations kind of language issues. so that is a great advantage. Going forward with the training, we would like to make the training more scalable, so we do more online videos, and classes, quiz kind of things online,people can actually do the training in smaller pieces, instead of losing full days,they can do it one day at a timeand do it in smaller pieces, and revisit it, sowe'd like to do more online training, but we also enjoy the classroom train, we would like to make themtougher and more specialized.
Graham:make it more hands on,so people can learn more.
Andrew: yes, make the classroom training harder,and the online training give them the foundation to come to the harder classes.
Xue: Beijing is the last city of this RWP China tour, so would you liketo share something related to the other two cities, Hangzhou and Shenzhen? It could be something like how many attendees were there, how did you feel there related to the reception of people with this event..
Andrew:没有雾霾!Ahh I think there about the same attendees as theBeijing class, again, same faces, you know. My biggest challenge we face is the language, I know some Chinese wordsnow
Graham: some words more than I do.
Andrew:I think the language, I must credit thepeople of China, when we first came here 11 years ago, very few people spokenEnglish and now, a lot of people speak English andreally well, which I think is fantastic.
Graham: we did try doing this one day, sessionswith translators, and it really didn't work very well, we have tried in several countries, we were basic blowing up translators, which is why we eventually said no, we will just doing it in English, because we had a lot of materials tocover, and doing it with translators, it's always got to slow it down,
So significantly, and even if you do synchronously translation, the content is so technical, it's really hardto get people who can translate that, so we decided that we just teach in English, as long as we tell the attendees, it's just going to be English before we start the training.
Xue: that's fine, probably it's the part we should work on, the other day, I came to the DevOpsdays conference,they uses this synchronously translator tool, I think they worked well. Maybewe can try some other time.
Graham: Graham: I don't think we have tried in China,we've tried in different countries, like Spanish, still, it didn't work very well. So we would have for the majority ofthe people that tried, give up with it.
Andrew:I guess the biggest thing for all theevents, is everyone is quite shy, I'm very lucky that I have a team in China,and I have got to know people here in China. I know people in China are not shy, they arevery happy to argue, and asking questions, it's just these events, people are still shy.
Xue:that's because their insecurity about their English.
Graham: yes.I think so.
Andrew: and to be honest, we don't care, we wish people would ask more questions and more be more animated, I guess if we knew the people for week by the end of the week, they would not be shy , but we only have one day.
Graham: this isn't just in China, pretty much everywhere we do this training ,where English was not the first language, there is always less questions, because people are embarrassed ,I think they are embarrassed by their English, we don't care.
Andrew:To be honest we are not embarrassed bythe Chinese people’s English, we are embarrassed by our lack of Chinese,every time I visit ,I try to learn a new word or expression
Xue: maybe you don't care, but still, they will be embarrassed, they do. Like me, right, talking to you ,I feel a little bit nervous.
Andrew:no worries, you are doing pretty good.
Xue:I assume you have been China for several times right? China is a country with very a very particular culture. Can you share with us something that could call the attention of you in this visit or before ?
Andrew: this trip is pretty much been abusiness, it's not much tourism, Graham and I just came to Beijing yesterday,but one of my fun trips is when Christine Qu and another Colleague we went to see the Great Wall, that was fantastic experience, I really enjoy during going up the GreatWall, and I'd like to visit it again.
Xue: so how long would you stay in Beijing?
Andrew: we are leaving this evening, we flight to Korea this evening.
Xue: well, anytime you wanna comeagain, China is always welcome you.
Andrew,Graham: thank you.
Graham: and I think to me, this time, came to Beijing, the incredible matter is growth is happening here, Beijing, and Hangzhou, Shenzhen, look completely different now, because they have been somuch incredible growth ,all the big new buildings , the city is changing remarkable very few years, something we don't see happening anywhere else, inother parts of the world
Xue: yes, some fast development happened. So the last time you mention that what shocked you the most is the age difference, do you have the same feeling now?
Graham: yes, we certainly see much younger people attending the event here than we see in the US, which is very good forus.
Andrew: I think it's fantastic, when we come to China it is the energyof the people, in everything, not just database, but in many things, energeticyoung people, in the hotel, airport ,at work ,clients, lot of young people,with lots of motivation, and it's very good. Lot of positive energy in China.
Xue:Regularly we share some technical articles related Oracle database through our company website orsocial media like WeChat, performancetuning is the most popular topic, but performance tuning is a big subject, so in your experience, what specific topic or topics ofperformance tuning call more the attraction of people ?
Andrew: it's interesting an subject,performance , I think there much you can read a lot about performance that makes very little difference in the real world. This which is strange, there are many people writing articles abouttopics that make 10% difference to the performance of a system, there are not many people writing article about making systems 100 times faster , 1000 times faster. This is one of the challenges and frustrations that Graham and I suffer. We often can see things that can make a system 1000 times faster, but we see people spend their time on things that make the system only 10% faster,。
We are just a very tiny team, within Oracle, we have 25 people, so along time ago , we decided we would focus on exclusively on the things that yield big gains. The things that make a system 1000 times faster. We research this and always to make a big difference, because for me, that is real world performance. It is frustrating alot of people who do in their jobs performance, don't actually make much difference to the performance, and the scary thing many people hired inperformance positions never improve performance , their job is just measure performance, they just make sure system can work functionally, but they don't make things faster. So I always encourage people to change and make things gofast, that at times can be difficult, Graham and my team, we focus on teaching people the way to identify where the big gains exist
Graham: right, that's the ways you identify the big gains, is bylooking at the data, we see people doing pattern matching, oh, we see this waitevent, let's see what people recommending, rather than taking a big picture to see what is going on in the system.
Xue:the top-down strategy
Graham: right, the top-down strategy, making sure you look top down,check the full operations that take place, rather than just focus on onecenter, to me, that's the key to do get the effective performance tuning.Understanding the root cause, and you can't do that without looking into the data, otherwise you are just guessing. We never want to be in the business of just guessing.
Xue: actually for many practical issues, the developers and operations they don't getalong, so sometimes even they think about to solve the problem from the application level, they can't, so instead of ,they just see what they can do tomake a little bit change, a little gain of the performance.
Graham: yes, you see, they don't want to change the performance. Tomused to say, they choose not to change the application. They can change the application , it's their application, they can change it, but they choose notto. Just focus on the little changes, which really can't make big performance gain.
Xue:I think it's given the issues between the departments
Andrew: yes, and particularly we appreciate this challenge in China,because it's a difficult communication. Nobody likes to be told their code is slow, because it's embarrassing,The programmer they may have done quite good job, based on the criterion whenthey wrote the code, the data may have grown a 1000 times bigger, and the codeis not appropriate for a big data set. In summary they may have done a good job 5 years ago, but times havechanged, again, people are very sensitive about the feedback and criticism,it's one of the challenges of the Real World Performance engineers to tellpeople the truth without upsetting them, and it's extremely difficult. Lookingat problems like an engineer we can use the data/statistics within a system totell us everything, so in many cases you just give them the numbers, and demonstrate how slow it is, and then people take a human reaction to it, they don't want to hear our message at times and this happens in every country ofthe world. Everywhere we go, nobody wants to change their code, however it isusually the problem, you have to attend a lot of meeting, win many arguments. So much of my teams trainingmaterial, demos, and videos is there to give people the evidence, if you changethe code correctly and how much faster the system would it go.
Graham: and one of the recent fashion in the IT industry, is DevOps,my hope for DevOps, is that helps to get the developers and DBA are actually working together. That's the idea rather than developers just generating thecode, and just throw it over the wall to the DBAs, who are then responsible for running well production. Instead, developers need to stay involved in the process and actually take responsibility for the code they generated. So DevOps could be really good, to help the systems get the performance they really need.
Xue: We actually have this kind of software, based on the concept of DevOps, named z3 SQL detection software,it aim to find the SQL statement that haven't be written well or may result inbig performance issues, and then the software will give the suggestions tochange it, or you can ask the expert to change it.
Graham: yes, great, so you can see the ability to change the SQL/code is more reasonable, with DevOps,we will see how it works.
Xue: the DevOps is not the new concept, it has come up for years, I'd say it works pretty well, just take timeto achieve it's value. So far, I find it works pretty good.
Graham: yes , lots of our customers start to adopt theDevOps, it is helping to breaking down that barrier of the system.
Andrew: There are many automatic tools today for the DBA. Many of their tasks like backups, space, allof these things are becoming more and more automatic, so DBA can focus their time on the performance,security and reliability of the system. So once the DBAs are not doing the repetitively tasks, and start workingon special projects, they effectively become the developers, because everyproject is a new project. In most cases,in most companies, people who actually know a lot of SQL are the DBAs, so theyshould be writing the SQL, and so it become more interesting for the DBAs intheir jobs. The other thing is about the organization of the company, when the developers and DBA are in the same project team, and they both getting the samebonus, there is lot more cooperation, So in many IT company, the developer teamand DBA team, they don't talk to each other, and the bonus completely different, but if they are all on the same project team, they both share the same bonus, they suddenly work a lot better together, so it is actually leadership and management behind this , not just technical aspects.
But if you have teams, all on the same project, suddenly there areless organization boundaries, that makes huge difference cause things can bedone really quickly if people cooperate.
Graham: yes, that's great, so you can see the ability to change the SQL/code is more reasonable, with DevOps,we will see how it works.
Xue: the DevOps is not the new concept, it has come up for years, I'd say it works pretty well, just take timeto achieve it's value. So far, I find it works pretty good.
Graham: yes , lots of our customers start to adopt the DevOps, it is helping to breaking down that barrier of the system.
Andrew: There are many automatic tools today for the DBA. Many of their tasks like backups, space, all of these things are becoming more and more automatic, so DBA can focus their time on the performance,security and reliability of the system. So once the DBAs are not doing the repetitively tasks, and start workingon special projects, they effectively become the developers, because every project is a new project. In most cases,in most companies, people who actually know a lot of SQL are the DBAs, so they should be writing the SQL, and so it become more interesting for the DBAs intheir jobs.
The other thing is about the organization of the company, when the developers and DBA are in the same project team, and they both getting the same bonus, there is lot more cooperation, So in many IT company, the developer team and DBA team, they don't talk to each other, and the bonus completely different, but if they are all on the same project team, they both share the same bonus, they suddenly work a lot better together,so it is actually leadership and management behind this , not just technical aspects.
But if you have teams, all on the same project, suddenly there are less organization boundaries, that makes huge difference cause things can bedone really quickly if people cooperate.
Graham: So Andrew talking about the remove the repetitive work from DBAs,and with move to Cloud, a big part of what DBA is doing now is done, if your database is running on the cloud,someone else would be responsible for running that system, then the company'sDBAs will become more like application DBAs, so they will understand more how applications use the database, and be able to providing the DevOps environment into how best to change the system to run better, rather than being focused on backups and storage.
Andrew: We at Oracle are defining the next platform in the cloud. In the Oracle cloud, the systems very standard high performance hardware and thereis little you can change on the platform. However if you want to fundamentally improve the reliability, the performance of a system it is going to come from the application algorithmic improvements and this is Real World Performance. The cloud revolution is making Real World Performance even more important. Real World Performance has more leverage in the cloud than we have in the on-promises model, so I am actually excited about the cloud because people will stop playing around for the platform issues, and focus on the applications design, and SQL, and this is where we know there is aorder of magnitude gains to performance ( 10X, 100X, 1000X ) . It's quite exciting to us when people stop focusing on the things that make very little gain and start focus on the things that makes big gains, so the cloud is exciting development for Real World Performance.
Xue: So do you think it's possible if all or the most of IT system move to the cloud, the company don'tneed their own operation DBAs anymore, if the system got problem, they maybejust need more professional team who provide the operation service like you, orin china, like us, to get the job done?
Andrew:For lot of operation staff, their jobs are quite boring, it's dull andrepetitive and people made mistakes. So you could automatic get these kinds of job you can move people to more innovative, creative positions so they can create better application more reliable, more secure application, they will have personal better careers and amore interesting career, and they will add more value to their companies.
So if people are more creative, they will enjoy their jobs more, so having very standard hardware, and software platforms and they are incredibly powerful platforms, in terms of IO, network, CPU, there is really no excuses for not writing good applications on those platforms. It puts the focus back to the creating interesting application, high-functional applications, secure applications, so it shift the innovation back to people working in the industry, and that has to be fun. The fun aspect is also from the good business point, because you creating the innovative products, business grows, so it's really important.
Graham: yes, it's very good, and about the application scalability,application in the cloud ,is likely to be good because of the hardware. Like Andrew said, the hardware in the cloud is incredible powerful, so the limitation you can see, is the applications, because they came back to findwhere the problem is, and getting that problem fixed in order to be able tobuild that application up.
Xue: about the hardware, youprobably seen that there is misunderstanding, or abuse, for many operation stuff, when they encounter the performance issue they can't figure out why or they don't bother to look into the problem, instead, they just change the hardware, if they are very lucky, they will get someperformance gain, it seems s much simple way to gain the performance a littlebit, but of course, they will meet the performance ceiling of the hardware.
Andrew: This is the one thing that you is touch is the real world issue, but people unfortunately, people refused to change their systems, and they don't like to hear about it, so they try to cover it up, by changing platform, or moving thing around, that's unfortunate, it's a human reaction,what we have found, is just like anything else in life. In motor sport the really good racing drivers,always win races and they get best out of their equipment, if you are not a good driver, you will not get the best out of your car. Todays Oracle Cloud environments are fantastic, it's time to explore them. Unfortunately, most people in IT system do not have competition like racing car drivers, where they have to go to the race to see who is the best, it's always vague, so people have areas they can make excuses, but when you going to a competition, you cannot make any excuses, you will win or loose, so these are some of the challenges we face in our working life. People will often deflect the real problem, rather than focus on thething that would make big differences, and this is human nature, I am afraid,there is nothing we can do about it. This is actually why Graham and I try to keep a good work life balance, because thereare lot of people who can go a lot faster but don't want to do the hard work,we don't get worried about these people.
Graham: they choose not to listen to us, wecan't do anything about it.
Andrew: yes, if people choose to be slow,that's their choice.
Xue: maybe it's not them don'twant to change, maybe it's their boss.
Andrew: yes, but you see, people can choose if they get average system, or poor system, for the rest of their life. You know. Leadership has important thing component to performance issues, so if the performance is very importantto the leadership, it tends to happen.
Xue:That's good point. You have lots of experience in performance tuning, What mediums, social, networks, blogs, books or another you use to share yourexperience ?
Graham: well we have serious of videos, and there is Chinese version which both has the English and Chinese subtitle on that, mainly Andrew talking through a number of demo we built over these years, and I think videos scales very well.
Andrew:We haven't written blogs, because thereare so many, so what we know about is writing blog is quite a job, you almost have to update it everyday, to be honest the quality of many of the blogs, it's pretty terrible. For one thing, I feel writing blog is more about the self promotion, rather that the promotion of the issues.
I am much more interested in the people understanding the engineeringconcepts, but not knowing my name, the most important thing is they understanding the engineering aspects, because that way they can really make a difference, I personally don't want to write books or blogs .
Xue :that's a good point, butstill I think book is pretty good way to spread the technology, you know,people can read them any time they want, or any place, they can even read themtime and time again, until they understand the topic, most ,they can mark inthe book, I think that's pretty good.
Graham: there are so many books around, and so many oracle books, in fact one of the things is that we all work at present to get the Real WorldPerformance approach into the Oracle documentation.
Andrew: so the oracle documentation of 12.2 has specific chapters ofthe Real World Performance, because we think that's the right place todocument, you know the official documentation should talk about these things,we would rather contribute there than write another book. Much of my team’s documentation gets copied and used. I take this copy as a form of compliment.
Xue:how about the experiences in your customers IT environment instead of your ideal system?
Graham: It's pretty much for every scenario, we have the demo to provehow to change to gain big performance, because usually people don't listen, weneed the data to show them why our recommendation is good idea and why theyhave to change it.
Andrew: one of the things I think my team has done a lot in last tenyears, is to build out the utility and tools, like Graham help design AWR, ASH, what we teach people to read and use these tools in different way, before people look into the utilities, and saw things to low level tuning, what we use this tools now is to identify the inefficiencies in a system and initiate a code change rather than change the setup of the database, so we use all the utilities to identify the design faultin the application. Many people are not aware when we look at the AWR report, it reveals the great deal about the how people administer their database and the shape of the application. We can understand what have gone on thesystem better than the people who running it. We are always finding new ways touse the database tools even if AWR is over ten years old now. We use these tools to identify theapplication design problems, rather than change the setup of the database we look to change the application code and get huge performance gains.
Graham: yes, using it as a way to understand the architecture of the application, across the multi-customers.
Andrew: yes, for many times, when the customers show their AWR reports, we identify something they didn't even know exist, like how their application is designed, in many cases, we ask questions like are you experiencing something in you system, they would often ask ‘How did you knowthat?’ In many cases we have seensimilar situations and we have predicted the failures before they have happened.
Graham: sometimes that's interesting ,the developers and DBAs, theDBAs, they say we can see through the AWR, you are gonna get this problem, but the developers was like, no no no,we don't have that problems. So they really should get in one room and looking at the system together, and see what is going on in the system. And in the afternoon ,we will talk about how to use AWR to identify you system, how we get things done.
Xue: Finally, if someone wants tobe become expert in performance tuning, what materials, what strategies. you can recommend to follow?
Graham: I would say, focus on the data, the data will lead you through detecting where the issues are, it's very easy for people to do, well I see some people they try to be experts in performance tuning, make sure you find the real problem before you try to fix them. The biggest mistake we have seen is that people throw in fixes to the things that might be the problems, they actually going to the wrong way. And back to the Andrew's point in this morning, make sure you are working on the things that have the biggest leverage, if you looking at the things are only 2or 3 percent of the time, no matter what you do to it, you are not going to make a big difference at all to the system ,you should work on things that are 50% , 60% to the time, obviously you can make a much bigger differences.
Andrew: and in addition to what Graham says, data driven, and focus is our philosophy, but at professional level if people want to progress their performance knowledge you have to do it all the time, so make sure you are working in a job that you are always doing performance projects. This really helps, and recognize that you can't always know everything,nobody will know everything, it's impossible, it's too big, but who surroundyourself, with good colleague and good mentors to coach you, and share ideaswith, I think that would be a big help for your career. There never been oneexpert who know everything, we often ask for second opinion when we get stuck,just recognize that hard work, and good mentoring, good communication, and let the data drive the process, is the key to professional develop.