가 (AWS, Python, DB, Linux, System, ELK, Ansible, Kafka, Java, C)

티스토리 뷰

창고

가 (AWS, Python, DB, Linux, System, ELK, Ansible, Kafka, Java, C)

yundal 2019. 8. 27. 02:12

[AWS]
#1 CloudFront에 static website를 올리는 방법
https://aws.amazon.com/ko/premiumsupport/knowledge-center/cloudfront-serve-static-website/
#2 Route53과 CloudFront 연동하기
http://pyrasis.com/book/TheArtOfAmazonWebServices/Chapter17/04
#3 NFS 설정하기
https://aws.amazon.com/ko/getting-started/tutorials/create-network-file-system/
위의 링크대로 한 후, 연결이 안될 경우 VPC default 보안그룹 설정 확인
#4 AWS Aurora
https://www.allthingsdistributed.com/2017/05/amazon-aurora-design-considerations.html

[Python]
#1 Log Parsing in Python
https://pythonicways.wordpress.com/2016/12/20/log-file-parsing-in-python/
#2 Python 정규표현식 (Regular Expression)
http://pythonstudy.xyz/python/article/401-%EC%A0%95%EA%B7%9C-%ED%91%9C%ED%98%84%EC%8B%9D-Regex
http://blog.naver.com/PostView.nhn?blogId=dudwo567890&logNo=130162403749
#3 Python Garbage Collection (GC) 의 원리
https://winterj.me/python-gc/
https://weicomes.tistory.com/277
#4 Python Garbage Collection - Instagram turned off the GC of Python
https://b.luavis.kr/python/dismissing-python-garbage-collection-at-instagram
#5 List Comprehension
[표현식 for 요소 in 컬렉션 [if 조건식]]
#6 Python Generator
https://item4.github.io/2016-05-09/Generator-and-Yield-Keyword-in-Python/
#7 Python 프로젝트 구조
https://trowind.tistory.com/105
https://python-guide-kr.readthedocs.io/ko/latest/writing/structure.html

[System]
#1 Flask와 Nginx 연동
https://blog.geuneul.com/archives/573
#2 Flask + Gunicorn + Nginx with Docker
https://medium.com/@kmmanoj/deploying-a-scalable-flask-app-using-gunicorn-and-nginx-in-docker-part-1-3344f13c9649
#3 Gunicorn Configuration / Gunicorn 설정 최적화
https://medium.com/building-the-system/gunicorn-3-means-of-concurrency-efbb547674b7
https://winterj.me/flask-concurrency-test/
#4 MongoDB to Elasticsearch
https://lifeinno.tistory.com/16
https://medium.com/@eldishnawy/tutorial-sync-mongodb-with-elasticsearch-fb43e9bc13ce

[Ansible]
#1 Ansible 기초
https://m.blog.naver.com/PostView.nhn?blogId=alice_k106&logNo=221333208746&proxyReferer=&proxyReferer=https%3A%2F%2Fwww.google.com%2F
#2 우아한 형제들 기술 블로그 Packer와 Ansible / 프로비저닝
http://woowabros.github.io/experience/2019/04/20/ami-packer-ansible.html
#3 Playbook
http://oniondev.egloos.com/9978477
https://moonstrike.github.io/ansible/2016/09/22/Ansible-Playbooks.html
https://docs.ansible.com/ansible/latest/user_guide/playbooks_best_practices.html?highlight=directory#directory-layout

[DB]
#1 Connection Pool 좋은 글
https://www.holaxprogramming.com/2013/01/10/devops-how-to-manage-dbcp/

[ELK]
#1 Elasticsearch Rejected Exception
https://brunch.co.kr/@alden/36
#2 검색엔진 좋은 글
https://needjarvis.tistory.com/167?category=704321
#3 Kafka to Logstash to Elasticsearch
https://gyrfalcon.tistory.com/entry/LogstashShipper-%EB%A5%BC-%EC%82%AC%EC%9A%A9%ED%95%B4-Kafka-%EB%A9%94%EC%84%B8%EC%A7%80%EB%A5%BC-elasticsearch%EB%A1%9C-%EC%A0%80%EC%9E%A5
#4 Logstash Kafka Input
https://yookeun.github.io/elasticsearch/2019/02/17/logstash-kafka/
#5 Logstash Drop Filter
https://cyuu.tistory.com/153
#6 Logstash JDBC Example - 베스트
https://dzone.com/articles/migrating-mysql-data-to-elasticsearch-using-logsta
#7 Logstash JDBC Setting
https://peung.tistory.com/13
#8 Elasticsearch Mapping
https://jjeong.tistory.com/854
#9 Elasticsearch Index Speed Tuning
https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html
https://wishgone.tistory.com/entry/%EC%97%98%EB%9D%BC%EC%8A%A4%ED%8B%B1%EC%84%9C%EC%B9%98-elasticsearch-%EB%A9%94%EB%AA%A8%EB%A6%AC-%EA%B4%80%EB%A0%A8-%EA%B3%B5%EC%8B%9D-%ED%99%88%ED%94%BC-%EB%B2%88%EC%97%AD
#10 Elasitcsearch 대용량 데이터 관리
https://www.slideshare.net/heungrae_kim/ss-123344339

[Kafka]
#1 Kafka 필수 설정
https://www.popit.kr/%EC%B9%B4%ED%94%84%EC%B9%B4-%EC%84%A4%EC%B9%98-%EC%8B%9C-%EA%B0%80%EC%9E%A5-%EC%A4%91%EC%9A%94%ED%95%9C-%EC%84%A4%EC%A0%95-4%EA%B0%80%EC%A7%80/?_fr=dable&utm_source=dable
#2 Kafka Cluster
https://epicdevs.com/20?category=460351
#3 Kafka 개념 및 원리
https://www.popit.kr/kafka-%EC%9A%B4%EC%98%81%EC%9E%90%EA%B0%80-%EB%A7%90%ED%95%98%EB%8A%94-%EC%B2%98%EC%9D%8C-%EC%A0%91%ED%95%98%EB%8A%94-kafka/
#4 Schema Registry
https://medium.com/@gaemi/kafka-%EC%99%80-confluent-schema-registry-%EB%A5%BC-%EC%82%AC%EC%9A%A9%ED%95%9C-%EC%8A%A4%ED%82%A4%EB%A7%88-%EA%B4%80%EB%A6%AC-1-cdf8c99d2c5c
#5 Kafka consumer list 명령어
https://stackoverflow.com/questions/32697999/kafka-consumer-list
#6 Kafka zookeeper option is deprecated, use bootstrap-server instead
https://stackoverflow.com/questions/53428903/zookeeper-is-not-a-recognized-option-when-executing-kafka-console-consumer-sh
#7 Kafka Offset Reset
https://www.letmecompile.com/kafka-consumer-offset-reset/
#8 Kafka Group Name
https://stackoverflow.com/questions/38549867/how-to-set-group-name-when-consuming-messages-in-kafka-using-command-line
#9 MongoDB Connector Option
https://github.com/mongodb/mongo-kafka/blob/master/docs/source.md
#10 Debezium MySQL Connector Configuration
https://rmoff.net/2018/03/24/streaming-data-from-mysql-into-kafka-with-kafka-connect-and-debezium/
https://support.etlworks.com/hc/en-us/articles/360020461693-Real-time-change-replication-with-Kafka-and-Debezium
https://knight76.tistory.com/entry/debezium-2
https://debezium.io/docs/tutorial/#start-kafka-connect
https://debezium.io/docs/connectors/mysql/#overview
#11 No suitable driver found 에러
https://www.confluent.io/blog/kafka-connect-deep-dive-jdbc-source-connector
#12 Kafka Producer 좋은 글
https://coding-start.tistory.com/193

[Command]
#1 Gunicorn SSL 사용
sudo gunicorn --certfile=경로 --keyfile=경로 -b 호스트:포트 -w 워커 수 app:app
#2 linux group에 user 추가
gpasswd -a 사용자 이름 그룹 이름
#3 JDK8 설치
sudo yum install java-1.8.0-openjdk-devel.x86_64 -y
#4 Grep 옵션
https://jhnyang.tistory.com/67
#5 hostname 설정
hostnamectl set-hostname abc
sudo reboot
#6 Shell Script 변수 조작
https://www.snoopybox.co.kr/1811
#7 vim 전체 선택
Shift + v + g