Feed aggregator
DevOpsDays Geneva – 2022
We started the DevOpsDays Geneva Thursday May 12th , with dbi services colleagues Pascal ZANETTE, Pierre-Yves BREHIER, Jean-Philippe CLAPOT and Chay TE, with the registration and a few cup of coffee, to prepare this first day.
After the Welcome speech provided by the event hosts, Matteo MAZZERI and Matthieu ROBIN, I follow the first main stream session, provided by Julia GIACINTI and Xavier NICOLOVICI, from PICTET, on “How to support the emergence of a DevOps culture within a large company”. In a well-defined sketch, Julie and Xavier explained the DevOps discovery of a traditional Prod manager and the incomprehension of this new way of work. They detailed this journey within their company, gave us a few tips and tricks, shared the challenges they faced and conclude with the current situation as well as the next steps.
It was then to David BARBARIN to present “Why we migrated the DB monitoring stack to Prometheus and Grafana”. David first detailed the current Migros Online architecture and explained the constraints and challenges which leads to the decision to use these tools. He gave a lots of details and explanation through his technical demonstration and conclude with the achievements and results of this migration.
The next presentation was done by Giovanni GALLORO, who gave a deep technical demonstration on Tekton pipelines named : “Tekton : from source to production inside Kubernetes”.
The last session of this busy morning was provided by Denis JANNOT who demonstrate how to implement “Advanced authentication patterns from the edge”. Denis shows the new challenges faced to properly secure a K8s cluster and detail several solution available which can be evaluate, based on infrastructure needs and constraints (Envoy proxy / Gloo Edge, API server…)
After a well deserved lunch break, we had a very interesting talk from Max ANDERSSON, who explain the audience how to “close the feedback loop for infrastructure development. Max recall the well know “3 ways” DevOps pillar and provide a refreshing and dynamic interactive session with the attendees.
We then reached the last session of this first by following Hervé SCHWEITZER, dbi services founder and CTO, session around YAK, a powerful internally developed tool for multi-cloud deployment. YAK is a derivation from IAC acronym (Infrastructure As Code) and was designed around Ansible and Docker, to allow a host deployment with the same setup independently of the destination (on prem AWS, Azure,…), using only one specific command. Hervé conclude with a Geneva DevOpsDays exclusive announcement : the YaK core component will be share with the community in the next months, probably around September this year. Stay tuned on this Blog website for the next announcements on the topic.
We ended this well filled first day with a cocktail diner, where passionate discussion and exchanges continued until late at night, in a very good ambiance.
After this short night (and a few cup of coffee), we started the second and last day of this event with Ankur MARFATIA, who clarify how to “Turn an enterprise to a learning community”. Ankur provide a real case example of learning share session put in place in his company, and underline the benefits of such internal events. He also linked these kind of internal practices with what can be found in external events, such as the DevOps Days. Interestingly, the importance of having food during these kind of meet-up was agreed by the whole assemble, which proves we are all the same.
He then explained some key points to create a safe and global learning environment for everyone, and reminds the two golden points for successful coaching sessions: Middle/Top Management implication and people wish to follow the training leaders.
Ankur conclude his talk with three important points to keep in mind :
1) learning is a never-ending journey
2) we all have different learning curves
3) change culture takes team and effort.
The next session was “Tips to fight the imposter syndrome”, by Aurélie VACHE a brilliant talk around this perception biases which leads a person to thing he/she does not belongs to his/her role or position. The person think his/her position is linked to luck and not hard work or knowledge. It leads to a self-deprecation feeling and the fear that other people will “realize” this imposture, soon or later. Aurélie gave a very frank and dynamic talk, with a lot of examples and tips to work around this syndrome and received a well deserved round of applause from everyone present in the room.
Right after, we listen carefully Courtney HIBA on how to “build mastery into your daily practice”. Courtney explained what was Mastery, why it was crucial for personal fulfillment and the different Mastery categories (Emotional, Business, Wealth and Relationship). She conclude by proposing an action plan to the audience : define one or two goals to master this year, identify the compelling reasons (why do you want to master it) and take action to start your mastery journey to achieve your goals.
Dr. Joe PEREZ followed, with a real show around “Driving decision with data : delight or disaster”. He gave a very energic talk, on the difference between the data’s values and their usage or pertinence. He gave us key points to help us to get enough materials to use data as accurately as possible, in order to improve our data-driven decisions, with all the necessary rules and safe guards to make it really useful and relevant.
We jumped into the “Docs-as-code : fix a poor document culture in your organization” presentation, dispensed by Sandro CIRULLI. Sandor first list the consequences of a poor documentation, which could leads to big issues such as : new employees onboarding slowness, productivity decrease, production outage recovery increased time, technical debt, new feature implementation slowness, poor communication,…
He then explained the documentation as code and why we should treat our documentation as we do for our code : versioned and trackable.
He shows the assembly a few tools which can be used for this purpose, such as Hugo, Red the Docs and Antora. He conclude that despite it need efforts from everyone at first, there is huge benefits to apply documentation-as-code and that we need to choose the tool that fits the company needs.
Stéphanie FISHER then share her personal experience and “lesson learned during an Agile transformation”. She highlights five key points she learned during her Agile coach career :
1st lesson : Avoid the word “Agile” to avoid people to block on a single word instead of embracing the concept and idea. 2nd : Resist the urge to fill the gaps : The risk is to substitute the root cause resolution against taking a role in the company. 3rd: Adapt yourself to the client context : you need to listen to your customer needs rather than pushing your own. 4th: Accept the frustration in the change : embrace the conflict if needed. Accept the tension as part of a necessary step in the change. 5Th : use your own advises and “Agile” yourself ! The world is changing and we need to be flexible, accept the incertitude, not over-analyze, prefer the testing to the thinking, prefer learning objective over that performance objective. It requires patience and resilience but it worst it.
Scott GRAFFIUS, remotely from the USA, detailed the Tuckman’s model to explain the five phases of a team development, Forming/Storming/Norming/Performing/Adjourning, and provide advice and guidance to help team members during those phases.
As a conclusion, Scott mentioned that all these steps are inevitable and seen in most of the teams, regardless of their activity or technical knowledge. Following the guidance shared will help team and individuals to prevent frustration and keep a good work spirit along the organization.
This presentation was followed by a participative open session with Matteo and Matthieu on what we thought about the organization of this event, if we had some improvement ideas or proposal, what we liked the most during these two days, the less,…
This conclude this DevOps Days Geneva and we hope we will see you there next year, we will be present for sure !
Cet article DevOpsDays Geneva – 2022 est apparu en premier sur Blog dbi services.
dbi services at the DevOpsDays Geneva 2022
dbi services was at the DevopsDays Geneva 2022 this year, which took place at the Haute Ecole de Gestion campus, in Carouge. Pierre-Yves Brehier, Pascal Zanette, Emmanuel Wagner, Chay Te and myself were present.
Opportunity to learn, discover new topics or technologies, doing in-person meetings and initiate business, this 2-days DevOps major event in the Romandie part of Switzerland gathered more than 400 people, 18 companies on site, and 16 additional sponsors.
With a mix of small dedicated rooms and more global sessions, this event covered a wide range of topics, technical or organizational.
Among all sessions we all attended to, here a small subset.
The first session I had the pleasure to attend was an interesting session Etienne Studer from Gradle who presented us the Developer Productivity Engineering.
This new engineering approach aim to increase developer productivity. Using automation and acceleration technologies, this approach is based on five pillars :
- Faster feedback cycles
- Faster Troubleshootings
- Reliable Builds and tests
- Continuous Learning and Improvement
- CI Cost and Resource efficiency
Another good session was the one performed by Giovanni Galloro, from Google Gloud, Specialist Customer Engineer.
He presented us Telkon. Open-source framework, Tekton is used to create CI/CD systems such as Jenkins, Jenkins X the one we use daily, Skaffold or more. In his session, he described all the basic components, such as Tasks, Pipelines, Steps, …
It was followed by a demo of the tool. Even if we knew it, it was very interesting to see Tekton, shown and explained by somebody from Google.
At the end of the first day, we were all there to support our CTO, Herve Schweitzer. He was presenting a new product from dbi services, YaK.
YaK, is a IaaC product. IaaC stands for an Infrastructure as a Code. It helps you to describe and deploy your infrastructure on specified targets. But in fact it’s way more : by using Ansible as a back-end but also all of our knowledge and expertise, it helps you to deploy virtual machines (Linux or Windows based) and databases (such as MariaDB, Postgres, Oracle, …) pain-free on Cloud providers (AWS, Azure or Oracle cloud for instance) or on-premise.
The beauty of that is changing where your DBs or VMs are deployed, so moving from one provider to another, is just a matter of setting the correct target! The room was packed for that session and the product and its concept was very well received by all the audience, concluded by a set of constructive discussions on the product itself and its application on the customer. dbi services is acting in favor of the community : a part of the product will be released public in September 2022 !
On the second day, we all attended in the amphitheater of the campus to a session of Aurelie Vache, from OVH Cloud.
With a warm support from the whole assistance, Aurelie didn’t talk about any technical topic here. No fancy DevOps tools here, “just” a talk about who we are: humans. Instead of being technical, she transported us into the discovery of a psychological pattern, called the impostor syndrome.
People affected feel that they don’t deserve their success or their position in a company. They feel the situation as if it were only due to chance. She gave some clues, like accepting we have knowledge, sharing and contributing, getting feedback, being positive. Thanks Aurelie for this very nice and refreshing presentation.
We can’t summarize this event to a set of sessions we attended or topics we had the pleasure to learn. It was also the opportunity, after last year cancellation, to meet people and share coffee or talks with customers, potential new employees or other DevOps fans like us !
Cet article dbi services at the DevOpsDays Geneva 2022 est apparu en premier sur Blog dbi services.
File Upload with PyScript
Systemd Persistent Logging
Remembrance from an old friend: Curiosity killed the cat, but satisfaction brought it back.
Finally found how to keep systemd logging persistent.
==================================================
### Number of reboots:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# last reboot
reboot system boot 5.4.17-2136.307. Sat May 21 17:53 - 18:01 (00:07)
reboot system boot 5.4.17-2136.307. Sat May 21 17:48 - 17:51 (00:03)
reboot system boot 5.4.17-2136.307. Sat May 21 11:01 - 17:48 (06:46)
reboot system boot 5.4.17-2136.307. Fri May 20 23:12 - 11:01 (11:48)
reboot system boot 5.4.17-2136.307. Fri May 20 22:54 - 23:11 (00:17)
reboot system boot 5.4.17-2136.307. Fri May 20 22:49 - 22:54 (00:04)
reboot system boot 5.4.17-2136.307. Fri May 20 22:21 - 22:49 (00:27)
reboot system boot 5.4.17-2136.307. Fri May 20 22:04 - 22:21 (00:16)
reboot system boot 5.4.17-2136.307. Thu May 19 23:35 - 22:03 (22:28)
reboot system boot 5.4.17-2136.307. Thu May 19 23:19 - 23:34 (00:15)
reboot system boot 5.4.17-2136.307. Thu May 19 23:07 - 23:18 (00:11)
reboot system boot 5.4.17-2136.307. Thu May 19 22:54 - 23:05 (00:11)
reboot system boot 5.4.17-2136.307. Thu May 19 22:47 - 22:53 (00:06)
reboot system boot 5.4.17-2136.307. Thu May 19 21:08 - 22:45 (01:37)
reboot system boot 5.4.17-2136.307. Thu May 19 20:40 - 21:07 (00:27)
reboot system boot 5.4.17-2136.307. Thu May 19 07:28 - 10:59 (03:31)
reboot system boot 5.4.17-2136.307. Thu May 19 07:22 - 07:27 (00:05)
reboot system boot 5.4.17-2036.100. Wed May 18 22:24 - 07:22 (08:57)
wtmp begins Wed May 18 22:24:31 2022
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### Only 1 reboot recorded from journalctl:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# journalctl --list-boots
0 016de73a7e81411bbd7c0b7657863852 Sat 2022-05-21 17:53:18 -04—Sat 2022-05-21 18:01:01 -04
[root@oracle-goldengate-1910-vagrant ~]#
[root@oracle-goldengate-1910-vagrant ~]# journalctl | head -1
-- Logs begin at Sat 2022-05-21 17:53:18 -04, end at Sat 2022-05-21 18:01:01 -04. --
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### Enable persistent systemd journal log:
==================================================
Enable persistent storage for the systemd journal log
https://gist.github.com/JPvRiel/b7c185833da32631fa6ce65b40836887
[root@oracle-goldengate-1910-vagrant ~]# mkdir /var/log/journal
[root@oracle-goldengate-1910-vagrant ~]# systemd-tmpfiles --create --prefix /var/log/journal
[root@oracle-goldengate-1910-vagrant ~]# echo "SystemMaxUse=100M" >> /etc/systemd/journald.conf
[root@oracle-goldengate-1910-vagrant ~]# grep -i SystemMaxUse /etc/systemd/journald.conf
#SystemMaxUse=
SystemMaxUse=100M
[root@oracle-goldengate-1910-vagrant ~]# systemctl restart systemd-journald.service
==================================================
### Moment of truth:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# last reboot
reboot system boot 5.4.17-2136.307. Sat May 21 18:31 - 18:33 (00:01)
reboot system boot 5.4.17-2136.307. Sat May 21 18:12 - 18:29 (00:16)
reboot system boot 5.4.17-2136.307. Sat May 21 17:53 - 18:09 (00:15)
reboot system boot 5.4.17-2136.307. Sat May 21 17:48 - 17:51 (00:03)
reboot system boot 5.4.17-2136.307. Sat May 21 11:01 - 17:48 (06:46)
reboot system boot 5.4.17-2136.307. Fri May 20 23:12 - 11:01 (11:48)
reboot system boot 5.4.17-2136.307. Fri May 20 22:54 - 23:11 (00:17)
reboot system boot 5.4.17-2136.307. Fri May 20 22:49 - 22:54 (00:04)
reboot system boot 5.4.17-2136.307. Fri May 20 22:21 - 22:49 (00:27)
reboot system boot 5.4.17-2136.307. Fri May 20 22:04 - 22:21 (00:16)
reboot system boot 5.4.17-2136.307. Thu May 19 23:35 - 22:03 (22:28)
reboot system boot 5.4.17-2136.307. Thu May 19 23:19 - 23:34 (00:15)
reboot system boot 5.4.17-2136.307. Thu May 19 23:07 - 23:18 (00:11)
reboot system boot 5.4.17-2136.307. Thu May 19 22:54 - 23:05 (00:11)
reboot system boot 5.4.17-2136.307. Thu May 19 22:47 - 22:53 (00:06)
reboot system boot 5.4.17-2136.307. Thu May 19 21:08 - 22:45 (01:37)
reboot system boot 5.4.17-2136.307. Thu May 19 20:40 - 21:07 (00:27)
reboot system boot 5.4.17-2136.307. Thu May 19 07:28 - 10:59 (03:31)
reboot system boot 5.4.17-2136.307. Thu May 19 07:22 - 07:27 (00:05)
reboot system boot 5.4.17-2036.100. Wed May 18 22:24 - 07:22 (08:57)
wtmp begins Wed May 18 22:24:31 2022
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### More than 1 reboot recorded from journalctl:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# journalctl --list-boots|sort
0 beedb88ca40c4c6187ae9d5c7ff1e96e Sat 2022-05-21 18:31:53 -04—Sat 2022-05-21 23:30:01 -04
-1 e12a9e02fd374350a5fcd3d4360b09fc Sat 2022-05-21 18:12:48 -04—Sat 2022-05-21 18:29:34 -04
-2 016de73a7e81411bbd7c0b7657863852 Sat 2022-05-21 17:53:18 -04—Sat 2022-05-21 18:09:16 -04
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### Review GoldengGate Start and Stop:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# last reboot|head -5|sort -n
reboot system boot 5.4.17-2136.307. Sat May 21 11:01 - 17:48 (06:46)
reboot system boot 5.4.17-2136.307. Sat May 21 17:48 - 17:51 (00:03)
reboot system boot 5.4.17-2136.307. Sat May 21 17:53 - 18:09 (00:15)
reboot system boot 5.4.17-2136.307. Sat May 21 18:12 - 18:29 (00:16)
reboot system boot 5.4.17-2136.307. Sat May 21 18:31 - 19:11 (00:39)
[root@oracle-goldengate-1910-vagrant ~]#
[root@oracle-goldengate-1910-vagrant ~]# journalctl --no-pager|grep "Manager st"
May 21 17:53:44 oracle-goldengate-1910-vagrant start_goldengate.sh[3115]: Manager started.
May 21 18:09:04 oracle-goldengate-1910-vagrant stop_goldengate.sh[6177]: Manager stopped.
May 21 18:13:15 oracle-goldengate-1910-vagrant start_goldengate.sh[3383]: Manager started.
May 21 18:29:21 oracle-goldengate-1910-vagrant stop_goldengate.sh[6290]: Manager stopped.
May 21 18:32:17 oracle-goldengate-1910-vagrant start_goldengate.sh[4134]: Manager started.
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
reboot system boot 5.4.17-2136.307. Sat May 21 17:48 - 17:51 (00:03)
journalctl May 21 17:53:44 oracle-goldengate-1910-vagrant start_goldengate.sh[3115]: Manager started.
reboot system boot 5.4.17-2136.307. Sat May 21 17:53 - 18:09 (00:15)
journalctl May 21 18:09:04 oracle-goldengate-1910-vagrant stop_goldengate.sh[6177]: Manager stopped.
journalctl May 21 18:13:15 oracle-goldengate-1910-vagrant start_goldengate.sh[3383]: Manager started.
reboot system boot 5.4.17-2136.307. Sat May 21 18:12 - 18:29 (00:16)
journalctl May 21 18:13:15 oracle-goldengate-1910-vagrant start_goldengate.sh[3383]: Manager started.
journalctl May 21 18:29:21 oracle-goldengate-1910-vagrant stop_goldengate.sh[6290]: Manager stopped.
reboot system boot 5.4.17-2136.307. Sat May 21 18:31 - 10:57 (16:25)
journalctl May 21 18:32:17 oracle-goldengate-1910-vagrant start_goldengate.sh[4134]: Manager started.
Adding a database to MySQL HeatWave using Auto Parallel Load
If you are reading this post, more than likely you are either looking at Oracle’s MySQL HeatWave for the first […]
The post Adding a database to MySQL HeatWave using Auto Parallel Load appeared first on DBASolved.
Systemd Database GoldenGate Service Start Stop
A complicated solution for a simple process.
If you can’t beat them, join them.
==================================================
### Restart vagrant box:
==================================================
resetlogs@ghost MINGW64 /e/oracle/vagrant-projects/OracleGoldenGate/19.3.0 (main)
$ vagrant halt
resetlogs@ghost MINGW64 /e/oracle/vagrant-projects/OracleGoldenGate/19.3.0 (main)
$ vagrant up
resetlogs@ghost MINGW64 /e/oracle/vagrant-projects/OracleGoldenGate/19.3.0 (main)
$ vagrant ssh
==================================================
### System info and last reboot:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# cat /etc/system-release
Oracle Linux Server release 7.9
[root@oracle-goldengate-1910-vagrant ~]#
[root@oracle-goldengate-1910-vagrant ~]# last reboot|head -1
reboot system boot 5.4.17-2136.307. Sat May 21 11:01 - 11:02 (00:00)
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### Verify goldengate.service started on reboot:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# systemctl status goldengate.service
● goldengate.service - Oracle GoldenGate Manager
Loaded: loaded (/etc/systemd/system/goldengate.service; enabled; vendor preset: disabled)
Active: active (running) since Sat 2022-05-21 11:02:14 -04; 1min 9s ago
Process: 4485 ExecStart=/home/oracle/scripts/start_goldengate.sh (code=exited, status=0/SUCCESS)
Main PID: 4504 (mgr)
CGroup: /system.slice/goldengate.service
├─4504 ./mgr PARAMFILE /u01/ogg/dirprm/mgr.prm REPORTFILE /u01/ogg/dirrpt/MGR.rpt PROCESSID MGR
└─4522 /u01/ogg/extract PARAMFILE /u01/ogg/dirprm/ext.prm REPORTFILE /u01/ogg/dirrpt/EXT.rpt PROCESSID EXT
May 21 11:02:06 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: Manager started.
May 21 11:02:06 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: GGSCI (oracle-goldengate-1910-vagrant) 3> START ER *
May 21 11:02:07 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: Sending START request to MANAGER ...
May 21 11:02:07 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: EXTRACT EXT starting
May 21 11:02:07 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: GGSCI (oracle-goldengate-1910-vagrant) 4> shell sleep 5
May 21 11:02:13 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: GGSCI (oracle-goldengate-1910-vagrant) 5> INFO ALL
May 21 11:02:14 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: Program Status Group Lag at Chkpt Time Since Chkpt
May 21 11:02:14 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: MANAGER RUNNING
May 21 11:02:14 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: EXTRACT RUNNING EXT 00:01:25 00:00:00
May 21 11:02:14 oracle-goldengate-1910-vagrant systemd[1]: Started Oracle GoldenGate Manager.
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### Check logs for goldengate.service:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# journalctl -u goldengate.service
-- Logs begin at Sat 2022-05-21 11:01:41 -04, end at Sat 2022-05-21 11:03:24 -04. --
May 21 11:02:04 oracle-goldengate-1910-vagrant systemd[1]: Starting Oracle GoldenGate Manager...
May 21 11:02:04 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: The Oracle base has been set to /opt/oracle
May 21 11:02:04 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: Oracle GoldenGate Command Interpreter for Oracle
May 21 11:02:04 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: Version 19.1.0.0.4 OGGCORE_19.1.0.0.0_PLATFORMS_191017.1054_FBO
May 21 11:02:04 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: Linux, x64, 64bit (optimized), Oracle 19c on Oct 17 2019 21:16:29
May 21 11:02:04 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: Operating system character set identified as UTF-8.
May 21 11:02:04 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: Copyright (C) 1995, 2019, Oracle and/or its affiliates. All rights reserved.
May 21 11:02:05 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: GGSCI (oracle-goldengate-1910-vagrant) 1>
May 21 11:02:05 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: GGSCI (oracle-goldengate-1910-vagrant) 2> START MGR
May 21 11:02:06 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: Manager started.
May 21 11:02:06 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: GGSCI (oracle-goldengate-1910-vagrant) 3> START ER *
May 21 11:02:07 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: Sending START request to MANAGER ...
May 21 11:02:07 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: EXTRACT EXT starting
May 21 11:02:07 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: GGSCI (oracle-goldengate-1910-vagrant) 4> shell sleep 5
May 21 11:02:13 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: GGSCI (oracle-goldengate-1910-vagrant) 5> INFO ALL
May 21 11:02:14 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: Program Status Group Lag at Chkpt Time Since Chkpt
May 21 11:02:14 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: MANAGER RUNNING
May 21 11:02:14 oracle-goldengate-1910-vagrant start_goldengate.sh[4485]: EXTRACT RUNNING EXT 00:01:25 00:00:00
May 21 11:02:14 oracle-goldengate-1910-vagrant systemd[1]: Started Oracle GoldenGate Manager.
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### There is no log from journalctl before reboot:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# last reboot|head -1
reboot system boot 5.4.17-2136.307. Sat May 21 11:01 - 11:32 (00:30)
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### NOTICE: Logs begin at Sat 2022-05-21 11:01:41
==================================================
[root@oracle-goldengate-1910-vagrant ~]# journalctl --no-pager|head -2
-- Logs begin at Sat 2022-05-21 11:01:41 -04, end at Sat 2022-05-21 11:30:01 -04. --
May 21 11:01:41 oracle-goldengate-1910-vagrant systemd-journal[151]: Runtime journal is using 8.0M (max allowed 96.8M, trying to leave 145.3M free of 960.7M available → current limit 96.8M).
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### stop_goldengate.sh attempts to gather info:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# ls -alrt /u01/ogg/ggse*
-rw-r-----. 1 oracle oinstall 7835 May 21 11:00 /u01/ogg/ggserr.log.20220521.1100
-rw-r--r--. 1 oracle oinstall 955 May 21 11:00 /u01/ogg/ggserr.log.stop
-rw-r-----. 1 oracle oinstall 10384 May 21 11:02 /u01/ogg/ggserr.log
[root@oracle-goldengate-1910-vagrant ~]#
[root@oracle-goldengate-1910-vagrant ~]# cat /u01/ogg/ggserr.log.stop
2022-05-21T11:00:47.915-0400 INFO OGG-00987 Oracle GoldenGate Command Interpreter for Oracle: GGSCI command (oracle): STOP MGR.
2022-05-21T11:00:47.930-0400 INFO OGG-00963 Oracle GoldenGate Manager for Oracle, mgr.prm: Command received from GGSCI on host [127.0.0.1]:58056 (STOP).
2022-05-21T11:00:47.930-0400 WARNING OGG-00938 Oracle GoldenGate Manager for Oracle, mgr.prm: Manager is stopping at user request.
2022-05-21T11:00:51.934-0400 INFO OGG-00987 Oracle GoldenGate Command Interpreter for Oracle: GGSCI command (oracle): STOP ER *.
2022-05-21T11:00:52.175-0400 INFO OGG-01021 Oracle GoldenGate Capture for Oracle, ext.prm: Command received from GGSCI: STOP.
2022-05-21T11:00:52.179-0400 INFO OGG-00991 Oracle GoldenGate Capture for Oracle, ext.prm: EXTRACT EXT stopped normally.
2022-05-21T11:00:59.213-0400 INFO OGG-00987 Oracle GoldenGate Command Interpreter for Oracle: GGSCI command (oracle): INFO ALL.
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### Continue with demo:
==================================================
==================================================
### Stop goldengate.service using systemctl stop:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# systemctl stop goldengate.service
[root@oracle-goldengate-1910-vagrant ~]# systemctl status goldengate.service
● goldengate.service - Oracle GoldenGate Manager
Loaded: loaded (/etc/systemd/system/goldengate.service; enabled; vendor preset: disabled)
Active: inactive (dead) since Sat 2022-05-21 11:16:37 -04; 13s ago
Process: 5432 ExecStop=/home/oracle/scripts/stop_goldengate.sh (code=exited, status=0/SUCCESS)
Process: 4485 ExecStart=/home/oracle/scripts/start_goldengate.sh (code=exited, status=0/SUCCESS)
Main PID: 4504 (code=exited, status=0/SUCCESS)
May 21 11:16:28 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Manager stopped.
May 21 11:16:28 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: GGSCI (oracle-goldengate-1910-vagrant) 3> STOP ER *
May 21 11:16:29 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Sending STOP request to EXTRACT EXT ...
May 21 11:16:30 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Request processed.
May 21 11:16:30 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: GGSCI (oracle-goldengate-1910-vagrant) 4> shell sleep 5
May 21 11:16:36 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: GGSCI (oracle-goldengate-1910-vagrant) 5> INFO ALL
May 21 11:16:37 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Program Status Group Lag at Chkpt Time Since Chkpt
May 21 11:16:37 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: MANAGER STOPPED
May 21 11:16:37 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: EXTRACT STOPPED EXT 00:00:00 00:00:07
May 21 11:16:37 oracle-goldengate-1910-vagrant systemd[1]: Stopped Oracle GoldenGate Manager.
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### Check journalctl -u goldengate.service:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# journalctl -u goldengate.service|grep -i stop
May 21 11:16:24 oracle-goldengate-1910-vagrant systemd[1]: Stopping Oracle GoldenGate Manager...
May 21 11:16:24 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: The Oracle base has been set to /opt/oracle
May 21 11:16:24 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: ‘/u01/ogg/ggserr.log’ -> ‘/u01/ogg/ggserr.log.20220521.1116’
May 21 11:16:24 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Oracle GoldenGate Command Interpreter for Oracle
May 21 11:16:24 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Version 19.1.0.0.4 OGGCORE_19.1.0.0.0_PLATFORMS_191017.1054_FBO
May 21 11:16:24 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Linux, x64, 64bit (optimized), Oracle 19c on Oct 17 2019 21:16:29
May 21 11:16:24 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Operating system character set identified as UTF-8.
May 21 11:16:24 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Copyright (C) 1995, 2019, Oracle and/or its affiliates. All rights reserved.
May 21 11:16:25 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: GGSCI (oracle-goldengate-1910-vagrant) 1>
May 21 11:16:25 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: GGSCI (oracle-goldengate-1910-vagrant) 2> STOP MGR !
May 21 11:16:26 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Sending STOP request to MANAGER ...
May 21 11:16:26 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Request processed.
May 21 11:16:28 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Manager stopped.
May 21 11:16:28 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: GGSCI (oracle-goldengate-1910-vagrant) 3> STOP ER *
May 21 11:16:29 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Sending STOP request to EXTRACT EXT ...
May 21 11:16:30 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Request processed.
May 21 11:16:30 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: GGSCI (oracle-goldengate-1910-vagrant) 4> shell sleep 5
May 21 11:16:36 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: GGSCI (oracle-goldengate-1910-vagrant) 5> INFO ALL
May 21 11:16:37 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: Program Status Group Lag at Chkpt Time Since Chkpt
May 21 11:16:37 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: MANAGER STOPPED
May 21 11:16:37 oracle-goldengate-1910-vagrant stop_goldengate.sh[5432]: EXTRACT STOPPED EXT 00:00:00 00:00:07
May 21 11:16:37 oracle-goldengate-1910-vagrant systemd[1]: Stopped Oracle GoldenGate Manager.
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### List ggserr logs:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# ls -alrt /u01/ogg/ggse*
-rw-r-----. 1 oracle oinstall 7835 May 21 11:00 /u01/ogg/ggserr.log.20220521.1100
-rw-r-----. 1 oracle oinstall 10384 May 21 11:16 /u01/ogg/ggserr.log.20220521.1116
-rw-r-----. 1 oracle oinstall 955 May 21 11:16 /u01/ogg/ggserr.log
-rw-r--r--. 1 oracle oinstall 955 May 21 11:16 /u01/ogg/ggserr.log.stop
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### Services configured for rdbms and goldengate:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# systemctl list-unit-files|egrep -i 'oracle|gate'
goldengate.service enabled
oracle-rdbms.service enabled
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### Configuration for goldengate.service:
==================================================
[root@oracle-goldengate-1910-vagrant ~]# ls -l /etc/systemd/system/goldengate.service
-rw-r--r--. 1 root root 713 May 19 23:16 /etc/systemd/system/goldengate.service
[root@oracle-goldengate-1910-vagrant ~]#
[root@oracle-goldengate-1910-vagrant ~]# cat /etc/systemd/system/goldengate.service
[Unit]
Description=Oracle GoldenGate Manager
After=syslog.target network.target
# systemctl list-unit-files|grep -i oracle
After=oracle-rdbms.service
[Service]
Type=forking
RemainAfterExit=yes
KillMode=none
User=oracle
Group=oinstall
TimeoutStopSec=60
TimeoutSec=300
Restart=no
# systemd ignores PAM limits, so set any necessary limits in the service.
# Not really a bug, but a feature.
# https://bugzilla.redhat.com/show_bug.cgi?id=754285
# /etc/security/limits.d/oracle-database-preinstall-19c.conf
LimitMEMLOCK=infinity
LimitNOFILE=131072
LimitNPROC=131072
ExecStart=/home/oracle/scripts/start_goldengate.sh
ExecStop=/home/oracle/scripts/stop_goldengate.sh
Restart=no
[Install]
WantedBy=multi-user.target
[root@oracle-goldengate-1910-vagrant ~]#
==================================================
### GoldenGate processes:
==================================================
oracle@oracle-goldengate-1910-vagrant ogg]$ ./ggsci
Oracle GoldenGate Command Interpreter for Oracle
Version 19.1.0.0.4 OGGCORE_19.1.0.0.0_PLATFORMS_191017.1054_FBO
Linux, x64, 64bit (optimized), Oracle 19c on Oct 17 2019 21:16:29
Operating system character set identified as UTF-8.
Copyright (C) 1995, 2019, Oracle and/or its affiliates. All rights reserved.
GGSCI (oracle-goldengate-1910-vagrant) 1> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EXT 00:00:00 00:00:00
GGSCI (oracle-goldengate-1910-vagrant) 2> info mgr detail
Manager is running (IP port TCP:oracle-goldengate-1910-vagrant.7809, Process ID 5539).
GGSCI (oracle-goldengate-1910-vagrant) 3> info extract *
EXTRACT EXT Last Started 2022-05-20 23:31 Status RUNNING
Checkpoint Lag 00:00:00 (updated 00:00:03 ago)
Process ID 5546
Log Read Checkpoint Oracle Integrated Redo Logs
2022-05-20 23:35:08
SCN 0.3722791 (3722791)
GGSCI (oracle-goldengate-1910-vagrant) 4> exit
[oracle@oracle-goldengate-1910-vagrant ogg]$
==================================================
### start_goldengate.sh & start_goldengate.obey:
==================================================
[oracle@oracle-goldengate-1910-vagrant ~]$ cat /home/oracle/scripts/start_goldengate.sh
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
ORAENV_ASK=NO; ORACLE_SID=ORCLCDB; OGG_HOME=/u01/ogg
. /usr/local/bin/oraenv
$OGG_HOME/ggsci << EOF
OBEY /home/oracle/scripts/start_goldengate.obey
exit
EOF
exit
[oracle@oracle-goldengate-1910-vagrant ~]$
[oracle@oracle-goldengate-1910-vagrant ~]$ cat /home/oracle/scripts/start_goldengate.obey
START MGR
START ER *
shell sleep 5
INFO ALL
[oracle@oracle-goldengate-1910-vagrant ~]$
==================================================
### stop_goldengate.sh & stop_goldengate.obey:
==================================================
[oracle@oracle-goldengate-1910-vagrant ~]$ cat /home/oracle/scripts/stop_goldengate.sh
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
ORAENV_ASK=NO; ORACLE_SID=ORCLCDB; OGG_HOME=/u01/ogg
. /usr/local/bin/oraenv
# Rotate existing GGSERR_LOG with append date
GGSERR_LOG=$OGG_HOME/ggserr.log
cp -fv ${GGSERR_LOG%%.*}.{${GGSERR_LOG#*.},${GGSERR_LOG#*.}."$(date +%Y%m%d.%H%M)"}
cat /dev/null > $GGSERR_LOG
$OGG_HOME/ggsci << EOF
OBEY /home/oracle/scripts/stop_goldengate.obey
exit
EOF
tail -9 $GGSERR_LOG > $GGSERR_LOG.stop
exit
[oracle@oracle-goldengate-1910-vagrant ~]$
[oracle@oracle-goldengate-1910-vagrant ~]$ cat /home/oracle/scripts/stop_goldengate.obey
STOP MGR !
STOP ER *
shell sleep 5
INFO ALL
[oracle@oracle-goldengate-1910-vagrant ~]$
==================================================
### GoldenGate credential store:
==================================================
GGSCI (oracle-goldengate-1910-vagrant) 1> INFO CREDENTIALSTORE
Reading from credential store:
Default domain: OracleGoldenGate
Alias: c##ggadmin
Userid: c##ggadmin@ORCLCDB
GGSCI (oracle-goldengate-1910-vagrant) 2>
==================================================
### GoldenGate mgr.prm:
==================================================
[oracle@oracle-goldengate-1910-vagrant ogg]$ cat dirprm/mgr.prm
PORT 7809
USERIDALIAS c##ggadmin
PURGEOLDEXTRACTS ./dirdat/*, USECHECKPOINTS, MINKEEPDAYS 3
AUTOSTART ER *
AUTORESTART ER *, RETRIES 5, WAITMINUTES 2, RESETMINUTES 60
CHECKMINUTES 5
LAGINFOMINUTES 0
LAGCRITICALMINUTES 5
[oracle@oracle-goldengate-1910-vagrant ogg]$
==================================================
### GoldenGate ext.prm:
==================================================
[oracle@oracle-goldengate-1910-vagrant ogg]$ cat dirprm/ext.prm
EXTRACT ext
SETENV (ORACLE_SID = "ORCLCDB")
SETENV (ORACLE_HOME = "/opt/oracle/product/19c/dbhome_1")
USERIDALIAS c##ggadmin
EXTTRAIL ./dirdat/et
GETUPDATEBEFORES
UPDATERECORDFORMAT COMPACT
SOURCECATALOG ORCLPDB1
TABLE ogguser.*;
SEQUENCE ogguser.*;
[oracle@oracle-goldengate-1910-vagrant ogg]$
==================================================
### tnsnames.ora requires (UR=A):
==================================================
ORCLPDB1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 0.0.0.0)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = ORCLPDB1)(UR=A)
)
)
==================================================
!!! IMPORTANT !!!
==================================================
systemctl is NOT aware when goldengate start/stopp from ggsci!
GGSCI (oracle-goldengate-1910-vagrant) 1> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EXT 00:00:00 00:00:09
GGSCI (oracle-goldengate-1910-vagrant) 2> stop mgr !
Sending STOP request to MANAGER ...
Request processed.
Manager stopped.
GGSCI (oracle-goldengate-1910-vagrant) 3> stop er *
Sending STOP request to EXTRACT EXT ...
Request processed.
GGSCI (oracle-goldengate-1910-vagrant) 4> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER STOPPED
EXTRACT STOPPED EXT 00:00:00 00:00:04
GGSCI (oracle-goldengate-1910-vagrant) 5>
==============================
[root@oracle-goldengate-1910-vagrant ~]# systemctl status goldengate.service
● goldengate.service - Oracle GoldenGate Manager
Loaded: loaded (/etc/systemd/system/goldengate.service; enabled; vendor preset: disabled)
Active: active (exited) since Sat 2022-05-21 12:00:46 -04; 1min 40s ago
Process: 5432 ExecStop=/home/oracle/scripts/stop_goldengate.sh (code=exited, status=0/SUCCESS)
Process: 7921 ExecStart=/home/oracle/scripts/start_goldengate.sh (code=exited, status=0/SUCCESS)
Main PID: 4504 (code=exited, status=0/SUCCESS)
May 21 12:00:37 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: GGSCI (oracle-goldengate-1910-vagrant) 2> START MGR
May 21 12:00:38 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: MGR is already running.
May 21 12:00:38 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: GGSCI (oracle-goldengate-1910-vagrant) 3> START ER *
May 21 12:00:39 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: EXTRACT EXT is already running.
May 21 12:00:39 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: GGSCI (oracle-goldengate-1910-vagrant) 4> shell sleep 5
May 21 12:00:45 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: GGSCI (oracle-goldengate-1910-vagrant) 5> INFO ALL
May 21 12:00:46 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: Program Status Group Lag at Chkpt Time Since Chkpt
May 21 12:00:46 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: MANAGER RUNNING
May 21 12:00:46 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: EXTRACT RUNNING EXT 00:00:00 00:00:00
May 21 12:00:46 oracle-goldengate-1910-vagrant systemd[1]: Started Oracle GoldenGate Manager.
[root@oracle-goldengate-1910-vagrant ~]#
==============================
GGSCI (oracle-goldengate-1910-vagrant) 1> sh date
Sat May 21 12:07:40 -04 2022
GGSCI (oracle-goldengate-1910-vagrant) 2> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER STOPPED
EXTRACT STOPPED EXT 00:00:00 00:05:45
GGSCI (oracle-goldengate-1910-vagrant) 3> start mgr
Manager started.
GGSCI (oracle-goldengate-1910-vagrant) 4> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EXT 00:00:00 00:00:03
GGSCI (oracle-goldengate-1910-vagrant) 5>
==============================
[root@oracle-goldengate-1910-vagrant ~]# systemctl status goldengate.service
● goldengate.service - Oracle GoldenGate Manager
Loaded: loaded (/etc/systemd/system/goldengate.service; enabled; vendor preset: disabled)
Active: active (exited) since Sat 2022-05-21 12:00:46 -04; 7min ago
Process: 5432 ExecStop=/home/oracle/scripts/stop_goldengate.sh (code=exited, status=0/SUCCESS)
Process: 7921 ExecStart=/home/oracle/scripts/start_goldengate.sh (code=exited, status=0/SUCCESS)
Main PID: 4504 (code=exited, status=0/SUCCESS)
May 21 12:00:37 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: GGSCI (oracle-goldengate-1910-vagrant) 2> START MGR
May 21 12:00:38 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: MGR is already running.
May 21 12:00:38 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: GGSCI (oracle-goldengate-1910-vagrant) 3> START ER *
May 21 12:00:39 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: EXTRACT EXT is already running.
May 21 12:00:39 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: GGSCI (oracle-goldengate-1910-vagrant) 4> shell sleep 5
May 21 12:00:45 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: GGSCI (oracle-goldengate-1910-vagrant) 5> INFO ALL
May 21 12:00:46 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: Program Status Group Lag at Chkpt Time Since Chkpt
May 21 12:00:46 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: MANAGER RUNNING
May 21 12:00:46 oracle-goldengate-1910-vagrant start_goldengate.sh[7921]: EXTRACT RUNNING EXT 00:00:00 00:00:00
May 21 12:00:46 oracle-goldengate-1910-vagrant systemd[1]: Started Oracle GoldenGate Manager.
[root@oracle-goldengate-1910-vagrant ~]#Removing a MySQL Heatwave Cluster
Recently I’ve been playing with Oracle’s MySQL Database Service (MDS) and testing out the Heatwave option. Heatwave is very impressive […]
The post Removing a MySQL Heatwave Cluster appeared first on DBASolved.
音楽専門学校で自分の進路を極める
音楽専門学校で自分の進路を極める
将来、音楽家を夢見て様々なアクションを起こす方は多いのですが、あらゆる知識を独学で習得するという事は困難な道でもあります。そういった悩みを抱えている場合は、音楽専門学校に通うという事が有効です。音楽専門学校においては、音楽に関する多くの学科が用意されており、進みたい道に応じて学科を選定することができるのです。一言で音楽業界と言っても様々なお仕事が存在しています。
ミュージシャンとして活躍する方が居る一方で、録音に携わるエンジニアやマネージメントに関わる業務など多岐にわたります。自分が進みたい道を見極めて、事前にしっかりと考慮した上で学ぶという意識を持つ事が重要になります。そして、音楽専門学校に進学すると現役でミュージシャンとして活動している方やエンジニアとして音楽制作に携わっている方の授業を受けることができます。現役で現場で働いている方のお話というのは、将来音楽業界に進みたいと考えている学生にとってはとても刺激的なものです。
自分の知識を増幅させることもでき、夢を実現させるために大きなステップアップを行うことができます。さらに、音楽専門学校ではアルバイトをしながら現場に通うという事が可能です。学生という身分でありながらも、本場の現場に早い段階から身を投じることにより、机上では学ぶことができない知識を得ることができるのです。当然、学校なので自分と同一の志を持っている方が多く仲間を作るといった事もできます。
Quiz Night
It’s a long time since I’ve done a quiz night – but here’s something that fooled me (briefly) when it appeared on the Oracle Developers’ Forum. Here’s a table definition – and I’m not going to make it easy by giving you a “create table” statement, but it’s just a simple heap table:
sql> desc interr_skuplannparam
name null? type
-------------------------------- -------- -------------------
atpdur number(38)
depdmdopt number(38)
externalskusw number(1)
firstreplendate date
lastfrzstart date
lastplanstart date
plandur number(38)
planleadtime number(38)
planleadtimerule number(38)
planshipfrzdur number(38)
restrictdur number(38)
allocbatchsw number(1)
cmpfirmdur number(38)
custservicelevel float(126)
maxchangefactor float(126)
mfgleadtime number(38)
recschedrcptsdur number(38)
cpppriority number(38)
cpplocksw number(1)
criticalmaterialsw number(1)
aggexcesssupplyrule number(38)
aggundersupplyrule number(38)
bufferleadtime number(38)
maxoh float(126)
maxcovdur number(38)
drpcovdur number(38)
drpfrzdur number(38)
drprule number(38)
drptimefencedate date
drptimefencedur number(38)
incdrpqty float(126)
mindrpqty float(126)
mpscovdur number(38)
mfgfrzdur number(38)
mpsrule number(38)
mpstimefencedate date
mpstimefencedur number(38)
incmpsqty float(126)
minmpsqty float(126)
shrinkagefactor number(38)
item varchar2(50 char)
loc varchar2(50 char)
expdate date
atprule number(38)
prodcal varchar2(50 char)
prodstartdate date
prodstopdate date
orderingcost float(126)
holdingcost float(126)
eoq float(126)
ff_trigger_control number(38)
workingcal varchar2(50 char)
lookaheaddur number
orderpointrule number
orderskudetailsw number(1)
supsdmindmdcovdur number(38)
orderpointminrule number(38)
orderpointminqty float(126)
orderpointmindur number(38)
orderuptolevelmaxrule number(38)
orderuptolevelmaxqty float(126)
orderuptolevelmaxdur number(38)
aggskurule number(38)
fwdbuymaxdur number(38)
costuom number(38)
cumleadtimedur number(38)
cumleadtimeadjdur number(38)
cumleadtimerule number(38)
roundingfactor float(126)
limitplanarrivpublishsw number(1)
limitplanarrivpublishdur number
maxohrule number(1)
integration_stamp date
integration_jobid not null varchar2(32 char)
error_str varchar2(2000 char)
error_stamp date
The column integration_jobid (the single “not null” column) has been defined with the default value of “INT_JOB”, which takes 7 bytes to store. What’s the result of the query at the end of this little script:
truncate table interr_skuplannparam;
insert into interr_skuplannparam (atpdur)
select 0
from all_objects
where rownum <= 10000
/
commit;
execute dbms_stats.gather_table_stats(user,'interr_skuplannparam')
select avg_row_len from user_tables;
Hint: the value zero is represented internally as a single byte holding the value 0x80 (decimal 128).
Helvetia used AWS SCT & DMS to migrate to AWS RDS for PostgreSQL
One of our long term-time customers, Helvetia, successfully migrated on-prem Oracle databases to AWS, not only because of the licenses, but more importantly: to deploy faster, innovate faster, and use the state-of-the-art open source database system.
When you plan such a project, you need to know which tools you want to use and what the target architecture shall look like. There are several options to choose from but finally Helvetia decided to use the AWS native services AWS DMS and AWS RDS for PostgreSQL.
AWS DMS gives you the option to initially populate the target instance from the source, and right afterwards logically replicates ongoing changes from the source to the target. However, before you can do that, you need the schema to be ready in the target. To prepare this, there is AWS SCT. This is not an AWS service, but a free tool you can use to convert a schema from a database system to another. If you want to go from Oracle to PostgreSQL, this tool also performs an automatic conversion from Oracle’s PL/SQL to PostgreSQL’s PL/pgSQL. Although this tool does a great job, you have to be very careful with the result, and invest a good amount of time in testing. Autonomous transactions, for example, do not exist in PostgreSQL and the AWS schema conversion utility implements a workaround using database links. This can be fine if you rarely use it (because it needs to establish a new connection) but if you rely heavily on this feature, you’d better re-implement in a way that is native to PostgreSQL.
Another area you need to pay attention to are the data types. PostgreSQL comes with many of them. A NUMBER in Oracle can mean many things in PostgreSQL. It could be an integer or a numeric in PostgreSQL. Depending on what you go for, this comes with space and performance impacts in PostgreSQL. PostgreSQL comes with a boolean data type. In Oracle, this is usually implemented as a character or a numeric value. Do you want to keep it that way or do you want to convert to a boolean? Converting means that you also need to adjust the business logic in the database.
Another issue that took quite some to solve was this. The very simplified test case attached to the initial Email showed massive performance drops in PostgreSQL compared to Oracle. The reason is that Oracle’s PL/SQL is a compiled language and PostgreSQL’s PL/pgSQL is interpreted. If you have a case that more or less matches what is described in the thread linked above, you need to re-write this. The same applies when you have commits or rollbacks in PL/SQL functions. PostgreSQL does not allow you to commit or rollback in a function. You need to use procedures for that.
These are just a few hints of what might come along the way when migrating to AWS RDS for PostgreSQL. Once you have solved all this, the migration can be really smooth and will most probably be a success. Here are some posts that describe how to set this up using an Oracle sample schema as the source:
- Migrating an Oracle instance to AWS Aurora – 1 – Setting up the base infrastructure
- Migrating an Oracle instance to AWS Aurora – 2– The AWS Schema Conversion Tool (SCT)
- Migrating an Oracle instance to AWS Aurora – 3 – Data replication with AWS DMS
- Migrating an Oracle instance to AWS Aurora – 4 – AWS DMS Change Data Capture (CDC)
If you follow that, you should have enough knowledge to get started with your journey to AWS RDS.
Cet article Helvetia used AWS SCT & DMS to migrate to AWS RDS for PostgreSQL est apparu en premier sur Blog dbi services.
Announcement: Dates Confirmed For Upcoming Webinars (“Here Today, Gone Tomorrow”)
音楽専門学校はサービスの内容を調査する
音楽専門学校はサービスの内容を調査する
深い専門知識を得る事ができる専門学校の中には、音楽に関係している所もあります。音楽専門学校に通学することで、通常の学習では学ぶ事が無い点も学ぶ事ができるようになります。そのため、音楽について興味がある方は迷わず音楽専門学校に進学を決めるのです。ニーズの高まりを受けて学校数は増加しており、行われる授業にも差異が現れるようになってきています。
こうした差異によって、これまで以上に自分の学習意欲や目的とマッチしている音楽専門学校を選ぶという事が重要になりました。つまり、進学する上では各々で異なるサービスについて知識を深める事が大切です。まず、大きな差異点として挙げられるのが講義の進め方についてです。一般的に専門学校で学ぶ場合は、大人数が収容できるスペースで勉強を進めていきますが、大人数の場合一人一人に講師がサポートすることができず、追いつくことができない生徒が出てきてしまうのです。
そこで、たくさんの人数で学ぶ事に苦手意識を抱いている方は少人数で授業が進められる音楽専門学校を探して学ぶと良いでしょう。少人数制であれば、周囲の人々に合わせるという必要が無いので焦ることなく落ち着いて自分のペースで学習を進めることができます。そして、音楽専門学校と言えば何よりも費用が心配になりますが、学費に関してしっかりとサポートしてくれる体制が整備されているので、金銭的に余裕が無いという場合でも諦める必要はありません。
Maximum number of Autonomous Databases on an ExaCC Full Rack
Number of cores before requring RAC licencing for BYOL
Automatic Indexes: Automatically Rebuild Unusable Indexes Part III (“Waiting For The Man”)
Connecting to MySQL Database Service (MDS) via DBeaver
With every new service on any cloud platform, the need to make connections is essential .This is the case with […]
The post Connecting to MySQL Database Service (MDS) via DBeaver appeared first on DBASolved.
A quick way of generating Informatica PowerCenter Mappings from a template
In our blogs we have discussed the options for Oracle Data Integrator (ODI) content generation here and here. Our go-to method is to use the ODI Java SDK, which allows querying, manipulating and generating new ODI content.
Can we do the same with Informatica PowerCenter? In the older PC versions there was the Design API that enabled browsing the repository and creating new content. However, I have never used it. My impression is that Oracle APIs are more accessible than Informatica APIs in terms of documentation, help available online and availability for download and tryout.
If we want to browse the PowerCenter repository content, there is an easy way - query the repository database. But what about content generation? Who will be brave or foolish enough to insert records directly into a repository database!? Fortunately, there is a way, and a fairly easy one, if you don't mind doing a bit of Python scripting.
Selective Informatica PC repository migrations are done via XML export and import - it is easy and mostly fool-proof. If we can generate XMLs for import, then we have found a way of auto-generating PowerCenter content. Informatica seems to support this approach by giving us nice, descriptive error messages if something is wrong with import XMLs. Only completely valid XMLs will import successfully. I have never managed to corrupt my Informatica repository with a dodgy XML import.
Let us look at an example - we need to extract a large number of OLTP tables to a Staging schema. The source and staging tables have very similar structures, except the staging tables have MD5 codes based on all non-key source fields to simplify change data capture (CDC) and also have the extract datetime.
- We start by creating a single mapping in Designer, test it, make sure we are 100% happy with it before proceeding further;
- We export the mapping in XML format and in the XML file we replace anything unique to the source and target table and their fields with placeholder tags:
[[EXAMPLE_TAG]]. (See the XML template example further down.) - Before we generate XMLs for all needed mappings, we need to import Source and Target table definitions from the databases. (We could, if we wanted, generate Source and Target XMLs ourselves but PC Designer allows us to import tables in bulk, which is quicker and easer than generating the XMLs.)
- We export all Sources into a single XML file, e.g.
sources.xml. Same with all the Targets - they go intotargets.xml. (You can select multiple objects and export in a single XML in Repository Manager.) The Source XML file will serve as a driver for our Mapping generation - all Source tables in thesources.xmlfile will have a Mapping generated for them. - We run a script that iterates through all source tables in the source XML, looks up its target in the targets XML and generates a mapping XML. (See the Python script example further down.) Note that both the Source and Target XML become part of the Mapping XML.
- We import the mapping XMLs. If we import manually via the Designer, we still save time in comparison to implementing the mappings in Designer one by one. But we can script the imports, thus getting both the generation and import done in minutes, by creating an XML Control File as described here.
Scripting Informatica PowerCenter Mapping generationA further improvement to the above would be reusable Session generation. We can generate Sessions in the very same manner as we generate Mappings.
The ImplementationAn example XML template for a simple Source-to-Staging mapping that includes Source, Source Qualifier, Expression and Target:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE POWERMART SYSTEM "powrmart.dtd">
<POWERMART CREATION_DATE="05/26/2021 11:55:12" REPOSITORY_VERSION="188.97">
<REPOSITORY NAME="DemoETL" VERSION="188" CODEPAGE="UTF-8" DATABASETYPE="Oracle">
<FOLDER NAME="Extract" GROUP="" OWNER="Developer" SHARED="NOTSHARED" DESCRIPTION="" PERMISSIONS="rwx---r--" UUID="55321111-2222-4929-9fdc-bd0dfw245cd3">
[[SOURCE]]
[[TARGET]]
<MAPPING DESCRIPTION ="[[MAPPING_DESCRIPTION]]" ISVALID ="YES" NAME ="[[MAPPING_NAME]]" OBJECTVERSION ="1" VERSIONNUMBER ="2">
<TRANSFORMATION DESCRIPTION ="" NAME ="SQ_EXTRACT" OBJECTVERSION ="1" REUSABLE ="NO" TYPE ="Source Qualifier" VERSIONNUMBER ="1">
[[SQ_TRANSFORMFIELDS]]
<TABLEATTRIBUTE NAME ="Sql Query" VALUE =""/>
<TABLEATTRIBUTE NAME ="User Defined Join" VALUE =""/>
<TABLEATTRIBUTE NAME ="Source Filter" VALUE =""/>
<TABLEATTRIBUTE NAME ="Number Of Sorted Ports" VALUE ="0"/>
<TABLEATTRIBUTE NAME ="Tracing Level" VALUE ="Normal"/>
<TABLEATTRIBUTE NAME ="Select Distinct" VALUE ="NO"/>
<TABLEATTRIBUTE NAME ="Is Partitionable" VALUE ="NO"/>
<TABLEATTRIBUTE NAME ="Pre SQL" VALUE =""/>
<TABLEATTRIBUTE NAME ="Post SQL" VALUE =""/>
<TABLEATTRIBUTE NAME ="Output is deterministic" VALUE ="NO"/>
<TABLEATTRIBUTE NAME ="Output is repeatable" VALUE ="Never"/>
</TRANSFORMATION>
<TRANSFORMATION DESCRIPTION ="" NAME ="EXPTRANS" OBJECTVERSION ="1" REUSABLE ="NO" TYPE ="Expression" VERSIONNUMBER ="2">
[[EXP_TRANSFORMFIELDS]]
<TRANSFORMFIELD DATATYPE ="nstring" DEFAULTVALUE ="ERROR('transformation error')" DESCRIPTION ="" EXPRESSION ="[[MD5_EXPRESSION]]" EXPRESSIONTYPE ="GENERAL" NAME ="CDC_MD5" PICTURETEXT ="" PORTTYPE ="OUTPUT" PRECISION ="32" SCALE ="0"/>
<TRANSFORMFIELD DATATYPE ="date/time" DEFAULTVALUE ="ERROR('transformation error')" DESCRIPTION ="" EXPRESSION ="SYSTIMESTAMP()" EXPRESSIONTYPE ="GENERAL" NAME ="EXTRACT_DATE" PICTURETEXT ="" PORTTYPE ="OUTPUT" PRECISION ="29" SCALE ="9"/>
<TABLEATTRIBUTE NAME ="Tracing Level" VALUE ="Normal"/>
</TRANSFORMATION>
[[SOURCE_INSTANCE]]
<INSTANCE DESCRIPTION ="" NAME ="SQ_EXTRACT" REUSABLE ="NO" TRANSFORMATION_NAME ="SQ_EXTRACT" TRANSFORMATION_TYPE ="Source Qualifier" TYPE ="TRANSFORMATION">
<ASSOCIATED_SOURCE_INSTANCE NAME ="[[SOURCE_INSTANCE_NAME]]"/>
</INSTANCE>
<INSTANCE DESCRIPTION ="" NAME ="EXPTRANS" REUSABLE ="NO" TRANSFORMATION_NAME ="EXPTRANS" TRANSFORMATION_TYPE ="Expression" TYPE ="TRANSFORMATION"/>
[[TARGET_INSTANCE]]
[[SRC_2_SQ_CONNECTORS]]
[[SQ_2_EXP_CONNECTORS]]
[[EXP_2_TGT_CONNECTORS]]
<CONNECTOR FROMFIELD ="CDC_MD5" FROMINSTANCE ="EXPTRANS" FROMINSTANCETYPE ="Expression" TOFIELD ="CDC_MD5" TOINSTANCE ="[[TARGET_INSTANCE_NAME]]" TOINSTANCETYPE ="Target Definition"/>
<CONNECTOR FROMFIELD ="EXTRACT_DATE" FROMINSTANCE ="EXPTRANS" FROMINSTANCETYPE ="Expression" TOFIELD ="EXTRACT_DATE" TOINSTANCE ="[[TARGET_INSTANCE_NAME]]" TOINSTANCETYPE ="Target Definition"/>
<TARGETLOADORDER ORDER ="1" TARGETINSTANCE ="[[TARGET_INSTANCE_NAME]]"/>
<ERPINFO/>
<METADATAEXTENSION COMPONENTVERSION ="1000000" DATATYPE ="STRING" DESCRIPTION ="" DOMAINNAME ="User Defined Metadata Domain" ISCLIENTEDITABLE ="YES" ISCLIENTVISIBLE ="YES" ISREUSABLE ="YES" ISSHAREREAD ="NO" ISSHAREWRITE ="NO" MAXLENGTH ="256" NAME ="Extension" VALUE ="" VENDORNAME ="INFORMATICA"/>
</MAPPING>
</FOLDER>
</REPOSITORY>
</POWERMART>Python script snippets for generating Mapping XMLs based on the above template:
- To translate database types to Informatica data types:
mapDataTypeDict = {
"nvarchar": "nstring",
"date": "date/time",
"timestamp": "date/time",
"number": "decimal",
"bit": "nstring"
}2. Set up a dictionary of tags:
xmlReplacer = {
"[[SOURCE]]": "",
"[[TARGET]]": "",
"[[MAPPING_DESCRIPTION]]": "",
"[[MAPPING_NAME]]": "",
"[[SQ_TRANSFORMFIELDS]]": "",
"[[EXP_TRANSFORMFIELDS]]": "",
"[[MD5_EXPRESSION]]": "",
"[[SOURCE_INSTANCE]]": "",
"[[SOURCE_INSTANCE_NAME]]": "",
"[[TARGET_INSTANCE]]": "",
"[[TARGET_INSTANCE_NAME]]": "",
"[[SRC_2_SQ_CONNECTORS]]": "",
"[[SQ_2_EXP_CONNECTORS]]": "",
"[[EXP_2_TGT_CONNECTORS]]": ""
}3. We use the Source tables we extracted in a single XML file as our driver for Mapping creation:
sourceXmlFilePath = '.\\sources.xml'
# go down the XML tree to individual Sources
sourceTree = ET.parse(sourceXmlFilePath)
sourcePowerMart = sourceTree.getroot()
sourceRepository = list(sourcePowerMart)[0]
sourceFolder = list(sourceRepository)[0]
for xmlSource in sourceFolder:
# generate a Mapping for each Source
# We also need to go down the Field level:
for sourceField in xmlSource:
# field level operations4. Generate tag values. This particular example is of a Column-level tag, a column connector between Source Qualifier and Expression:
sqToExpConnectorTag = f'<CONNECTOR FROMFIELD ="{columnName}" FROMINSTANCE ="SQ_EXTRACT" FROMINSTANCETYPE ="Source Qualifier" TOFIELD ="{columnName}" TOINSTANCE ="EXPTRANS" TOINSTANCETYPE ="Expression"/>'5. We assign our tag values to the tag dictionary entries:
xmlReplacer["[[SQ_2_EXP_CONNECTORS]]"] = '\n'.join(sqToExpConnectors)6. We replace the tags in the XML Template with the values from the dictionary:
for replaceTag in xmlReplacer.keys():
mappingXml = mappingXml.replace(replaceTag, xmlReplacer[replaceTag])Interested in finding out more about our approach to generating Informatica content, contact us.
Data Annotation with SVG and JavaScript
Pages
